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PREFACE 


Our  research  program  was  engendered  by  puzzlement  over  why  the  recommendations  of  so 
many  studies  on  reforming  the  intelligence  community  have  resulted  in  so  few  changes.  We  are 
certainly  not  the  first  to  notice  this  state  of  affairs.  One  observer  examined  all  of  the  studies  of 
U.S.  intelligence  between  1991  and  2001  and  found  that  80%  of  the  recommendations  made  in 
those  studies  were  not  implemented.  More  recently  the  question  has  been  raised  that  if  there  has 
been  so  much  attention  for  so  long  on  fixing  intelligence,  why  does  it  never  seem  to  get  fixed? 
The  most  frequent  explanations  for  the  lack  of  progress  towards  reform  of  the  intelligence 
community  include  bureaucratic  resistance,  organizational  aversion  to  change,  and  resistance  to 
changes  that  address  new  challenges. 

Our  research  program  posed  a  different  question  about  the  reason  for  the  lack  of  implementation 
of  the  recommendations.  Our  question  is  “Does  the  existing  knowledge  and  understanding  of 
intelligence  analysis,  as  manifested  in  the  literature  of  intelligence  analysis,  inform  the 
development  and  application  of  both  the  mechanistic  and  cognitive  activities  needed  to  support 
doing  intelligence  analysis?”  Our  answer,  after  an  intensive,  five-year  analysis  of  the 
intelligence  analysis  literature  and  related  scientific  literature,  is  no. 

We  identified  the  following  shortcomings  in  the  intelligence  analysis  literature: 

•  The  description  of  the  analysis  process,  and  the  means  for  improving  it,  are  largely 
cast  in  tenns  of  what  an  analyst  should  do.  However,  very  little  guidance  can  be 
found  in  the  literature  that  tells  analysts  how  to  do  what  they  should  do,  particularly 
with  regards  to  the  cognition  involved  with  analysis. 

•  The  literature  contains  a  heavy  focus  on  the  end  phase  of  the  analysis  process,  that  is, 
the  phase  associated  with  coming  up  with  the  “answer”  to  an  intelligence  question. 

As  a  result,  important  work  that  must  be  done  in  the  preceding  intelligence  analysis 
phases  such  as  defining  the  problem,  acquiring  information,  determining  the  meaning 
of  information,  and  the  transformation  of  information  into  evidence  receives  much 
less  attention. 

•  While  Heuer  made  notable  contributions  30  years  ago  by  integrating  the  results  of 
cognitive  psychology  into  the  intelligence  analysis  process,  little  use  has  been  made 
of  any  subsequent  research  findings  in  the  field  that  might  also  help  understand  the 
analysis  process.  Most  notable  is  the  lack  of  use  of  the  findings  on  System  1  and 
System  2  modes  of  thinking. 

•  Analysts  are  told  to  avoid  bias.  Sometimes  the  bias  to  be  avoided  is  cognitive  bias, 
that  is,  a  departure  from  rationality.  In  other  instances  the  bias  to  be  avoided  is  a 
judgment  that  is  considered  unreasoned  and  shows  a  lack  of  objectivity.  Other  than 
telling  analysts  to  avoid  bias,  the  literature  is  silent  with  regards  to  the  means  of  doing 
so. 

Based  on  our  assessment  of  both  the  intelligence  literature  and  areas  of  scientific  research  related 
to  analytical  cognition  we  concluded  that  the  need  existed  for  the  development  of  a  body  of 
thought  that  covers  all  of  the  essential  elements  of  intelligence  analysis  from  end  to  end,  that 
provided  a  comprehensive  and  flexible  framework  supporting  intelligence  analysts  doing  their 
work.  The  Framework  for  Analytic  Cognition  (FAC)  is  our  response  to  this  need. 


IX 


The  choice  of  the  word  framework  is  deliberate.  FAC  is  neither  a  manual  nor  a  handbook  but  a 
description  of  a  basic  conceptual  structure.  FAC  is  designed  to  provide  an  end-to-end  guide  for 
doing  intelligence  analysis  that  is  systematic,  yet  flexible  enough  to  accommodate  the 
unstructured  and  highly  iterative  nature  of  the  work.  FAC  focuses  on  aiding  the  analyst  in 
developing  a  thorough,  deep  understanding  of  both  capabilities  and  intentions  as  shaped  by 
human  decision  and  actions  for  their  subject  of  study. 

In  addition,  our  intent  is  that  FAC  address  the  shortcomings  in  the  intelligence  literature  that  we 
identified  above.  FAC  places  particular  emphasis  on  the  issues  of  bias  and  objectivity. 

Extensive  research  shows  that  Type  1  thinking,  sometimes  called  System  1  thinking,  is  the 
preferred  mode  of  thinking  but  is  the  source  of  nearly  all  cognitive  biases.  Type  2  thinking  is  a 
more  deliberative  mode  of  thinking  in  which  one’s  beliefs  and  knowledge  strongly  affect  the 
results  of  thinking.  The  results  of  this  research  show  that  Type  2  thinking  has  to  override  Type  1 
thinking  in  order  to  have  thinking  largely  devoid  of  cognitive  biases.  The  use  of  Type  2 
thinking,  however,  results  in  thinking  that  is  integral  to  the  individual.  It  will  be  subjective  to 
some  and  objective  to  others.  The  underlying  intellectual  principle  used  in  the  construction  of 
FAC  is  the  requirement  for  Type  2  thinking  to  override  Type  1  thinking.  Such  overrides  require 
deliberate  effort  and  recognition  of  the  need  by  the  individual  to  do  so.  The  means  used  in  FAC 
to  foster  such  overrides  is  epistemic  writing:  writing  that  not  only  results  from  thought  but 
writing  that  becomes  an  integral  part  of  thought  in  conjunction  with  all  cognitive  actions.  The 
written  product  also  provides  a  record  of  the  information,  knowledge,  and  beliefs  used  to  develop 
the  results  of  the  cognition  so  one  individual  can  compare  their  subjectivity  to  the  subjectivity  of 
the  writer. 

This  initial  version  of  FAC  was  written  with  the  practicing  intelligence  analyst  as  its  audience. 
FAC  is  written  as  a  guide  for  analysts  to  pick  up  and  use.  It  does  not  prescribe  a  mechanistic 
approach  to  analysis  or  “the”  way  of  doing  analysis.  FAC  describes  cognition  and  the  inputs  to 
each  cognitive  act  starting  with  a  need  for  an  intelligence  judgment  to  the  act  of  communicating 
the  response  to  that  need.  By  design,  FAC  is  careful  not  to  constrain  the  cognitive  and  creative 
aspects  of  this  very  unstructured  work.  Consequently,  the  use  of  structured  processes  and  aids 
such  as  checklists  are  avoided.  We  have  endeavored  to  provide  enough  structure  to  guide  the 
work,  while  maintaining  the  flexibility  for  the  individual  to  apply  FAC  given  his  or  her  personal 
styles  of  work  and  thinking. 

FAC  has  been  read  and  put  into  use  by  a  small  number  of  analysts  and  their  managers.  To  date 
the  response  has  been  a  few  suggested  changes  amid  very  positive  comments  on  FAC’s  content 
and  utility.  We,  the  authors,  also  know  of  changes  to  be  made.  None  of  our  changes,  nor  those 
proposed  by  analysts  and  their  mangers,  alter  the  basic  conceptual  content  of  FAC.  We  believe 
the  dialogue  about  analytic  cognition  must  be  broadened,  so  FAC  is  being  made  widely 
available.  The  intended  audience  for  FAC  includes: 

•  educators  in  university  intelligence  programs 

•  educators  in  intelligence  organizations 

•  commentators  and  students  of  intelligence  analysis  in  the  policy  community, 
academia,  and  private  sector 

•  those  informing  development  of  computer  assisted  support  for  analysts 

•  those  developing  and  applying  the  analytic  competencies  of  analysts  and  their 
managers 


x 


FAC  is  a  work  in  progress.  In  FAC,  we  say  that  once  an  analyst  has  formed  an  answer  to  an 
intelligence  question,  the  analyst  needs  to  start  at  the  beginning  again  since  the  analyst  is  a 
different  person,  intellectually,  than  when  the  analysis  was  initiated.  This  guidance  is  equally 
applicable  to  the  authors  of  FAC  -  so  we  are  starting  over  again.  Your  comments  on  FAC  are 
desired  and  needed  to  help  us  further  improve  FAC. 


1.0  SUMMARY 


FAC  is  a  comprehensive  guide  for  doing  intelligence  analysis.  It  is  designed  to  provide  an  end- 
to-end  guide  for  doing  intelligence  analysis  that  is  systematic,  yet  flexible,  to  accommodate  the 
unstructured  and  highly  iterative  nature  of  intelligence  analysis  work.  FAC  focuses  on  aiding 
intelligence  analysts  in  developing  a  thorough,  deep  understanding  of  both  capabilities  and 
intentions  as  shaped  by  human  decision  and  actions  for  their  subject  of  study. 

The  essence  of  intelligence  analysis  is  detennining  the  originator’s  meaning  from  information, 
and  FAC  is  designed  to  aid  in  detennining  that  meaning.  The  FAC  contains  the  following 
essential  elements  to  aid  analysts  in  determining  meaning  and  doing  intelligence  analysis.  These 
essential  elements  are  represented  in  the  sections  of  the  FAC. 

•  Understanding  self,  beliefs 

•  Writing  to  think 

•  Exploring  purpose,  issue  framing,  understanding  actual  or  prospective  customer  needs 

•  Planning  analysis  substance,  collaboration,  resources  &  timing 

•  Acquiring  information,  systematically  and  thoroughly 

•  Assigning  meaning  to  infonnation,  individually  and  in  the  aggregate 

•  Preparing  an  analysis  representation  separate  from  the  customer  product 

•  Communicating  the  assessment  and  its  significance  to  customers 

FAC  leverages  existing  knowledge  relevant  to  intelligence  analysis  from  potentially  applicable 
scientific  and  professional  subject  areas.  In  addition,  FAC  is  being  reviewed  and  used  by  a  small 
number  of  intelligence  analysts.  Their  feedback  to  date  has  been  positive.  Based  on  our  research 
and  this  feedback,  we  believe  that  the  FAC  is  now  usable  by  analysts.  However,  FAC 
development  is  a  work  in  progress.  Additional  research  synthesis  is  being  done  to  further 
improve  the  FAC,  and  additional  research  studies  are  planned  to  evaluate  the  FAC. 
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2.0  INTRODUCTION 

2.1  The  Genesis 

Starting  with  Pearl  Harbor,  the  U.S.  intelligence  community  has  often  faced  criticism  for  failing 
to  predict  or  warn  of  future  events.  Numerous  examples  of  these  failures  have  been  cited.  Two 
of  the  most  recent  and  highly  publicized  examples  include  the  failure  to  warn  against  the  9-11 
attack  and  incorrectly  estimating  Iraq’s  weapons  program.  As  a  result,  numerous  groups  and 
commissions  have  been  established  during  the  past  30  years  to  study  “what  went  wrong.”  All  of 
these  efforts  have  resulted  in  proposals  to  “fix”  the  intelligence  community  and  thereby  improve 
the  quality  of  its  products.  While  the  proposals  have  come  from  different  groups,  a  certain 
commonality  exists  among  the  proposals. 

As  one  might  expect,  one  of  the  proposed  solutions  is  an  increase  in  resources.  However,  both 
critics  and  supporters  of  the  intelligence  community  believe  that  the  solution  does  not  only  lie  in 
providing  more  resources.  Instead,  there  is  a  reoccurring  theme  that  the  core  reason  for  the 
failure  of  analysis  is  “faulty  thinking”  on  the  part  of  the  analyst.  From  this  perspective,  the 
necessary  quality  of  the  intelligence  product  will  only  be  achieved  by  improvements  in  both  the 
quality  and  productivity  of  analysts.  In  this  context,  past  and  present  solutions  proposed  have 
included  selecting  more  qualified  personnel,  enhancing  training,  developing  new  methodologies 
for  analysis,  and  organizing  analytical  efforts.  In  the  last  several  decades,  proposals  for  the 
improvement  of  intelligence  analysis  have  also  emphasized  the  need  to  develop  automated  tools 
to  aid  in  the  acquisition  and  processing  of  information,  as  well  as  tools  that  will  enhance 
analysis. 

The  fact  that  the  same  solutions  are  proposed  time  and  time  again  suggests  a  failure  of 
implementation.  This  failure  could  stem  from  many  causes,  but  a  potential  cause  of  particular 
concern  is  whether  the  requisite  knowledge  for  the  implementation  of  the  enhancements  exists. 
For  example,  do  we  know  the  cognitive  and  personality  attributes  (e.g.,  extrovert  or  introvert)  of 
a  “good  analyst?”  What  does  an  analyst  need  to  know  in  order  to  function  effectively?  What  are 
the  mechanistic  as  well  as  the  cognitive  processes  associated  with  the  acquisition  and  processing 
of  data  or  information?  What  are  the  characteristics  of  the  cognitive  transfonnation  of  data  or 
information  into  useable  knowledge? 

It  is  reasonable  and  logical  to  assume  that  answers  to  these  questions  exist  within  the  intelligence 
community.  Certainly,  the  subject  of  intelligence  analysis  has  not  been  ignored.  A  large  number 
of  books,  blogs,  reports,  white  papers,  and  articles  on  intelligence  analysis  exist,  and  new 
publications  are  added  to  the  literature  each  year.  However,  even  though  a  great  deal  of 
intelligence  literature  exists,  does  the  literature  contain  all  the  requisite  knowledge  to  implement 
intelligence  analysis  enhancements  and  improvements?  Is  the  reoccurrence  of  the  same 
proposals  for  improvement  due  solely  to  a  failure  to  put  this  existing  knowledge  into  practice  or 
is  it  due  to  a  lack  of  information  in  the  literature?  Future  efforts  to  improve  the  analytical 
process  are  unlikely  to  be  successful  until  these  questions  can  be  answered. 

Air  Force  Research  Laboratory  (AFRL)  initiated  research  to  gain  knowledge  and  understanding 
of  intelligence  analysis  as  described  in  published  literature  and  documents.  This  publication, 
Framework  for  Analytic  Cognition  (FAC),  is  the  result  of  that  research  effort  to  answer  the  first 
question:  Does  the  existing  intelligence  analysis  literature  contain  all  the  knowledge  needed  to 
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implement  intelligence  analysis  improvements  and  enhancements?  The  approach  used  in  this 
study  included  the  following: 

•  Acquire  published  literature  regarding  all  aspects  of  intelligence  analysis  and  related 
areas  of  science,  particularly  cognitive  psychology. 

•  Assess  the  accumulated  knowledge  about  intelligence  analysis  in  terms  of  its 
completeness,  consistency,  and  compatibility  with  knowledge  in  related  scientific 
areas. 

•  Analyze  the  acquired  information  in  the  context  of  intelligence  analysis,  the 
characteristics  of  intelligence  analysis,  and  the  attributes  of  the  individuals 
performing  intelligence  analysis. 

•  Develop  recommendations  relative  to  the  attributes  and  skills  necessary  for  quality 
analysis  and  relative  to  the  development  of  analytic  methodologies  and  work  support 
aids  and  tools. 

Based  on  the  review  and  assessment  of  over  3,000  documents,  this  research  effort  provided  an 
answer  to  the  question  of  whether  the  existing  literature  on  intelligence  analysis  contains  the 
requisite  knowledge  to  inform  the  development  and  application  of  both  the  mechanistic  and 
cognitive  activities  to  support  doing  intelligence  analysis.  The  answer  is  no.  In  marked 
contrast,  the  results  of  scientific  research  on  cognition  (and  in  many  other  scientific  fields)  do 
provide  an  enhanced  understanding  of  the  analysis  process  and  the  factors  that  affect  analysis. 
Some  of  the  intelligence  literature  shows  an  awareness  of  this  applicable  cognitive  research,  but 
the  application  of  the  results  to  intelligence  analysis  is  flawed.  Indeed,  some  of  the  conventional 
wisdom  in  the  intelligence  literature  is  contradictory  to  what  is  known  and  accepted  in  cognitive 
research,  and  we  will  elaborate  on  this  throughout  the  FAC. 

Not  only  is  the  intelligence  literature  lacking  in  the  knowledge  necessary  to  support  the 
development  of  effective  processes  for  analysis,  in  some  instances  the  lack  of  knowledge  has  a 
negative  effect.  Some  of  the  methods  and  procedures  for  conducting  analysis  described  in  the 
literature  can  contribute  to  invalid  analytic  assessments.  As  we  will  elaborate  in  the  FAC, 
deficiencies  include: 

•  Rare  recognition  that  information  retrieval  is  a  necessary  skill  for  intelligence 
analysts. 

•  Using  a  proposed  answer  or  a  set  of  proposed  answers  to  the  intelligence  question  or 
issue  as  the  starting  point  for  intelligence  analysis. 

•  No  recognition  of  the  potential  for  conflicts  arising  from  personal  and  organizational 
beliefs  during  the  intelligence  analysis. 

•  Not  fully  addressing  the  important  role  of  ever  changing  context  and  knowledge  in 
the  analyst’s  determination  of  meaning  of  information. 

2.2  FAC  Purpose 

One  recommendation  from  this  research  effort  was  to  develop  a  framework  that  covers  all  of  the 
essential  elements  of  doing  intelligence  analysis  from  end  to  end,  providing  a  comprehensive  and 
flexible  framework  based  on  scientific  findings  to  support  intelligence  analysts  doing  their  work. 
The  FAC  is  the  implementation  of  that  recommendation  and  provides  the  requisite  knowledge 
based  on  the  existing  science.  The  FAC  focuses  on  supporting  an  analyst  addressing  intelligence 
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questions  and  issues  in  his  or  her  area  of  responsibility,  including  those  self-initiated  and  those 
posed  by  others. 

Traditionally  an  intelligence  analyst  has  been  assigned  a  specific  area  of  responsibility;  for 
example:  a  scientific  discipline,  a  technology,  a  weapon  system,  individuals  of  interest,  or  a 
geographic  area.  The  analyst  is  then  expected  to  gain  comprehensive  knowledge  and 
understanding  of  foreign  activities,  developments,  and  intentions  in  his  or  her  area  of 
responsibility  and  to  produce  authoritative  products  based  on  this  knowledge.  However,  while 
an  analyst  may  have  sole  responsibility  to  generate  assessments  for  his  or  her  area  of 
responsibility,  he  or  she  does  not  work  in  isolation  since  developments  in  areas  that  are  the 
responsibility  of  other  analysts  will  affect  developments  in  his  or  her  area  of  responsibility. 
Analysts  may  work  with  others  who  contribute  to  the  assessment  or  the  assessment  may  be 
developed  in  collaboration  with  others.  Most  assuredly,  analysts  will  have  others  who  review 
and  impose  changes  to  the  content  of  their  assessments.  Thus,  the  ultimate  quality  of  the 
intelligence  assessment  and  the  resultant  product  depends  on  the  input  of  many  individuals. 

The  situation  described  above  illustrates  a  basic  paradox  of  analysis.  In  one  context,  formulating 
an  assessment  is  a  collaborative  endeavor,  but  ultimately  an  assessment  must  be  put  forth  by  an 
individual.  The  individual  analyst  is  both  a  recipient  of  the  assessments  of  others  and  a  provider 
to  others  to  aid  in  their  fonnulation  of  assessments.  While  the  role  of  collaboration  is  important, 
the  purpose  of  the  FAC  is  not  to  prescribe  the  collaborative  approach  to  be  used  by  all 
individuals  involved  in  creating  an  intelligence  assessment.  However,  the  FAC  does  provide 
approaches  that  enhance  collaboration,  e.g.,  through  planning  for  the  needed  collaboration  and 
capturing  the  analyst’s  thinking  in  writing  to  share  during  collaboration.  The  FAC  focuses  on 
the  actions  and  analysis  of  the  analyst  responsible  for  addressing  intelligence  questions  in  his  or 
her  area  of  responsibility,  including  self-initiated  issues  and  issues  posed  by  others.  The  FAC 
does  not  address  areas  such  as  coordination,  review,  and  approval  processes. 

The  objective  of  the  FAC  is  to  stimulate  your  thinking  by  providing  information  on  the  analytical 
methods  and  approaches  that  will  maximize  the  quality  of  the  resulting  assessment.  In  addition, 
the  FAC  provides  a  foundation  for  collaboration  during  generation  of  the  intelligence  assessment 
or  required  answer,  as  well  as  during  subsequent  quality  reviews  of  the  product. 

2.3  Definitions 

What  is  intelligence  analysis ?  Based  on  our  literature  review,  there  is  no  agreed  upon  definition 
or  purpose.  This  lack  of  an  agreed  upon  definition  and  purpose  is  hardly  surprising  given  the 
lack  of  agreement  on  the  term  intelligence.  The  recent  efforts  of  Warner,1  Johnston,2  and 
Wheaton  and  Beerbower3 4  to  establish  a  definition  of  intelligence  are  part  of  an  ongoing  effort. 

Many  of  the  definitions  of  intelligence  encompass  specific  aspects  of  intelligence  analysis. 
Intelligence  analysis  has  been  described  in  tenns  of  the  sources  and  classifications  of 
information,  processes,  purposes,  individual  and  organizational  efforts,  and  consumers. 

Common  themes  include  the  use  of  single  or  all  sources  of  information,  use  of  denied  or  secret 


1  Warner,  M.  "Wanted:  A  Definition  of  "Intelligence","  Studies  in  Intelligence,  Vol.  46,  No.  3,  2002,  pp.  15-22 

2  Johnston,  R.  Analytic  Culture  in  the  US  Intelligence  Community:  An  Ethnographic  Study,  Center  for  the  Study  of 
Intelligence,  Washington,  D.  C.,  2005. 

3  Wheaton,  K.J.  &  Beerbower,  M.T.  "Towards  a  New  Definition  of  Intelligence,"  Stanford  Law  and  Policy  Review, 
Vol.  17,  No.  2,  2006,  pp.  319-330. 
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information  (although  most  find  this  too  limiting),  discerning  pertinent  information,  making 
inferences  and  judgments,  individually  and  collectively  developing  knowledge  of  current  and 
potential  threats  to  national  security,  and  providing  useable  and  actionable  information  for  the 
military  and  government  and  non-govemment  policymakers. 

The  definitional  problem  has  been  compounded  by  the  growing  use  of  the  term  intelligence  to 
describe  many  fields  such  as  business  intelligence  and  criminal  intelligence.  In  addition,  more 
confusion  is  added  since  analysis  is  a  function  associated  with  numerous  endeavors  like 
economics,  history,  and  political  and  social  sciences. 

The  basic  question  is  whether  these  other  intelligence  and  analysis  activities  are  different  than 
those  associated  with  “military”  or  “national”  intelligence.  From  a  cognitive  standpoint  the 
answer  is  no.  The  distinction  among  all  of  these  intelligence  endeavors  is  who  does  the  analysis 
and  who  is  the  intended  customer  of  the  analysis  products.  For  the  purpose  of  the  FAC, 
intelligence  and  analysis  are  what  is  done  by  the  organizations  under  the  purview  of  the  Director 
of  National  Intelligence. 

Most  descriptions  of  the  functions  and  purpose  of  intelligence  analysis  also  contain  elements  that 
are  consistent  with  the  standard  definitions  of  both  analysis  and  synthesis  (e.g.,  Webster’s  New 
World  Dictionary,  Third  College  Edition): 

•  “ Analysis',  a  separating  or  breaking  up  of  any  whole  into  its  parts,  esp.  with  an 
examination  of  these  parts  to  find  out  their  nature,  proportion,  function, 
interrelationship,  etc.” 

•  “ Synthesis :  the  putting  together  of  parts  or  elements  so  as  to  form  a  whole” 

However,  these  descriptions  of  the  intelligence  analysis  process  often  contain  elements  that  go 
beyond  the  above  definitions,  such  as  requirement  definition,  collection,  and  communicating  the 
results.  For  the  purpose  of  this  handbook,  the  tenn  intelligence  analysis  process  encompasses  all 
the  activities  from  exploring  the  question  or  issue  to  be  addressed  through  providing  the 
intelligence  assessment.  The  term  intelligence  analysis  is  used  to  describe  the  activities  that  are 
inherent  in  the  dictionary  definition  of  analysis  and  synthesis. 

The  question  “What  is  intelligence  analysis?”  is  further  confused  by  the  many  different  ways  that 
intelligence  analysis  is  categorized.  Some  categorize  according  to  the  types  of  intelligence 
questions  addressed  by  intelligence  analysis.  These  questions  are  often  referred  to  according  to 
the  following  commonly  used  categories: 

•  Strategic 

•  Tactical 

•  Current 

•  Indications  and  Warning 

Analysis  is  also  often  categorized  as  either  single  source  or  all  source. 

Another  category  of  intelligence  analysis  exists,  but  is  often  not  discussed  in  the  literature.  This 
category  concerns  the  physical  nature  of  the  target  of  interest.  For  example,  some  aspects  of 
such  analysis  involve  determining  the  characteristics  of  a  signal  or  the  technical  performance  of  a 
system  and  subsystems,  determining  radar  cross  sections  or  infrared  signatures,  measuring  object 
sizes,  or  tracking  numbers  of  things.  The  common  attributes  of  such  analysis  are  that  it  is 
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dependent  on  technical  or  engineering  knowledge  and  judgments  about  physical  placement  or 
parameters,  without  consideration  of  the  role  of  the  human  in  the  development  or  activity. 

In  contrast  with  the  above,  another  category  is  focused  on  the  understanding  of  human  decisions 
and  actions.  It  attempts  to  answer  questions  such  as  “What  has  been  done?”,  “Why  has  it  been 
done”,  “What  was  the  intent  of  the  actions?”,  “What  are  the  ramifications  of  the  actions?”,  “How 
does  it  affect  their  capabilities?”,  and  “What  do  those  capabilities  indicate  about  their 
intentions?” 

The  FAC  focuses  on  understanding  both  capabilities  and  intentions  as  shaped  by  human 
decisions  and  actions.  Across  all  of  these  categories,  a  common  characteristic  exists  in  all  of 
these  types  of  analysis.  That  is,  intelligence  analysis  attempts  to  discover  what  a  foreign  nation 
or  adversary  doesn’t  want  to  be  known.  In  other  words,  the  core  objective  of  intelligence 
analysis  is  to  discern  what  is  unknown  to  us:  hidden  or  classified  plans,  capabilities,  and 
intentions  of  a  foreign  nation  or  adversary,  i.e.,  their  secrets. 

2.4  FAC  Contents 

While  disagreement  might  exist  on  the  definition  and  purpose  of  intelligence  analysis,  the  belief 
that  intelligence  analysis  is  cognitively  highly  complex  and  demanding  is  universally 
acknowledged.  The  typical  intelligence  analyst  is  confronted  with  a  substantial  amount  of 
information.  The  fact  that  some  information  is  missing  and  some  is  deliberately  misleading  only 
complicates  the  job  of  the  analyst.  From  the  available  information,  true  or  not,  the  analyst  must 
determine  meaning  from  information  in  order  to  develop  knowledge  and  understanding  so  that 
intelligence  questions  can  be  answered. 

Determining  meaning  and  developing  knowledge  and  understanding  are  not  accomplished 
through  a  single  cognitive  process.  Intelligence  analysis,  from  start  to  finish,  relies  on  many, 
interdependent  cognitive  processes.  These  cognitive  processes  include  multiple  inputs  and 
decisions  throughout  the  intelligence  analysis.  The  results  of  these  processes  are  highly 
dependent  on  the  analyst  involved.  For  this  reason,  an  analyst  should  know  what  he  or  she  adds 
to  the  determination  of  meaning  throughout  the  analysis.  To  help  analysts  understand  this,  the 
next  section  of  the  FAC  discusses  analysts  as  individuals  and  their  impact  on  cognitive 
processes.  This  “Analysts  as  Individuals”  section  of  the  FAC  includes  the  following  topics: 

•  Thinking  About  Thinking.  The  assignment  of  meaning  to  information  is  dependent 
on  the  individual’s  past  experiences,  education,  cultural  values,  knowledge,  beliefs, 
etc.  The  section  discusses  how  the  mind  functions,  types  of  cognitive  propensities, 
and  role  of  knowledge,  beliefs  and  goals.  All  of  these  factors  influence  your 
assignment  of  meaning  to  information  and  shape  your  behaviors  and  judgments. 

•  Knowing  Yourself.  The  concept  of  humans  as  rational  decision  makers  has  been 
thoroughly  discounted  in  the  last  decade.  Recent  research  in  neuroscience  and 
cognitive  psychology  has  shown  consistent  behavior  that  one  author  has  termed 
“predictably  irrational.”4  In  other  words,  this  behavior  is  predictable  except  by  the 
individual  so  behaving.  These  human  behaviors  affect  many  aspects  of  the 
interaction  between  individuals  and  information,  as  well  as  the  interactions  with  other 
individuals.  An  analyst  should  have  awareness  of  such  behaviors  so  that  he  or  she 


4  Ariely,  D.  Predictably  Irrational:  The  Hidden  Forces  that  Shape  our  Decisions,  Harper,  New  York,  NY,  2008. 
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can  adopt  practices  to  mitigate  some  of  these  behaviors.  Since  many  human 
behaviors  can  affect  the  analysis  process  and  the  resultant  product,  an  awareness  of 
these  behaviors  and  one’s  susceptibility  to  such  behaviors  is  appropriate. 

•  Writing  As  Thinking.  This  topic  discusses  why  writing  is  an  essential  underlying 
element  of  the  overall  analysis  process.  Because  writing  helps  stimulate  your 
thinking  and  cognition,  we  discuss  why  it  is  important  to  document  or  keep  an 
external  representation  of  your  thinking,  as  it  occurs  throughout  each  element  of  the 
FAC.  This  external  representation  can  be  used  to  capture  your  thinking  about  a  topic, 
manipulate  those  thoughts  to  create  new  thoughts,  engage  in  self-reflection  about 
your  thinking,  and  share  your  thinking  with  others. 

The  rest  of  the  FAC  describes  each  essential  element  of  the  intelligence  analysis  process, 
including  inputs  and  decisions,  and  their  sequential  and  iterative  relationships.  The  essential 
elements  of  the  process  described  are: 

•  Developing  Intelligence  Issue  Statements.  The  meaning  of  information  is  very 
dependent  on  why  one  is  making  the  judgment.  This  section  covers  approaches  for 
thoroughly  exploring  and  understanding  the  intelligence  question  or  issue  being 
addressed.  If  the  question  is  not  self-initiated  and  comes  from  someone  else  (e.g.,  a 
customer),  the  approaches  described  provide  the  means  to  ensure  that  the  customer 
and  the  analyst  have  the  same  understanding  of  the  need  and  purpose  for  the 
intelligence  analysis. 

•  Setting  a  Direction:  The  Analysis  Plan.  This  section  describes  approaches  for 
planning  the  overall  analysis  effort  required  to  address  the  intelligence  issue.  By 
exploring  how  potential  events  associated  with  the  target  of  your  intelligence  issue 
may  unfold,  you  can  identify  more  specifically  the  substantive  areas  that  you  will 
need  to  address.  It  will  help  you  think  about  who  may  be  involved  in  your  target’s 
decisions  and  why  their  actions  may  be  taking  place.  This  exploration  helps  you 
initially  scope  the  substantive  areas  to  be  addressed,  including  the  areas  where  you 
may  need  analytic  contributions  from  others.  The  section  also  covers  planning 
activities,  time,  resources,  and  support  required  to  accomplish  the  analysis  in  the  time 
available. 

•  Developing  the  Information  Plan.  In  this  FAC  section,  you  translate  your 
understanding  of  the  intelligence  issue,  as  well  as  the  events,  participants,  and 
decisions  described  by  the  substantive  areas  identified  in  the  analysis  plan,  into  a 
strategy  for  seeking  information  about  those  topics.  This  aids  you  in  determining 
possible  sources  of  information,  the  information  repositories  to  be  searched,  who  will 
conduct  the  necessary  searches,  and  an  overall  information  acquisition  schedule  that 
summarizes  your  information  acquisition  strategy. 

•  Information  Acquisition.  This  section  describes  how  to  develop  search  strategies 
based  your  Information  Plan  into  appropriate  queries  for  the  repositories  to  be  used. 
The  section  also  addresses  executing  and  keeping  track  of  your  searches,  reviewing 
the  results  of  searches,  judging  the  relevance  of  the  information,  managing  the 
relevant  information  acquired,  formulating  new  search  queries,  and  the  iterative 
nature  of  information  acquisition. 
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•  Analysis  Techniques.  Determining  the  meaning  of  information  is  essential  to 
analysis.  We  provide  infonnation  in  this  FAC  section  to  assist  you  with  developing 
and  focusing  your  mental  framework  in  preparation  for  this  crucial  aspect.  We 
elaborate  upon  many  of  the  concepts  introduced  in  the  “Analysts  As  Individuals, 
Thinking  About  Thinking”  section.  Information  is  also  provided  on  the  role  of 
structured  analytic  techniques  (SATs)  in  both  the  intelligence  literature  and  how 
they  relate  to  the  cognition  of  analysis. 

•  Assignment  of  Meaning.  All  information  relevant  to  the  intelligence  issue  under 
study  is  evidence  of  something.  The  challenge  is  to  detennine  “evidence  of  what?” 
Finding  out  what  others  do  not  want  you  to  discover  is  central  to  seeking 
infonnation  and  developing  and  using  the  analytic  approaches  discussed  in  this 
section.  The  first  step  in  finding  the  unknown  is  to  assign  meaning  to  each  of  the 
individual  pieces  of  relevant  information  that  resulted  from  your  infonnation 
acquisition  efforts.  Focused  approaches  for  acquiring  context  and  answer  questions 
about  what  and  why  are  provided.  In  addition,  key  concepts  that  are  the  foundation 
for  the  analysis  approaches  embedded  in  FAC  are  discussed. 

•  Assessment.  At  this  stage  you  have  finished  assigning  meaning  to  all  the 
infonnation  that  you  judged  as  being  relevant  to  the  intelligence  issue  under 
analysis.  You  have  developed  tentative  hypotheses,  searched  for  more  information, 
and  developed  more  tentative  hypotheses.  The  objective  covered  in  this  section  is 
now  to  transform  your  tentative  hypotheses  into  beliefs  about  the  intelligence  issue. 
The  section  discusses  marshalling  your  infonnation  and  beliefs  for  your  assessment, 
and  using  your  knowledge  of  the  current  status  to  predict  what  will  result  in  the 
future. 

•  Analysis  Representation.  For  reasons  elaborated  in  the  “Analysts  as  Individuals, 
Writing  as  Thinking”  section,  analysts  must  represent  in  writing  the  cognitive 
aspects  of  their  analysis  for  each  element  of  the  FAC  throughout  the  analysis 
process.  This  section  discusses  bringing  all  of  these  representations  together  into  an 
integrated  form.  The  Analysis  Representation  represents  the  totality  of  your 
analytic  efforts. 

•  The  Customer  Product.  This  section  covers  communicating  your  assessment  to  a 
customer  or  audience  in  a  written  product.  It  discusses  analyzing  your  audience, 
using  the  analysis  representation  to  develop  an  effective  narrative,  and  conveying 
your  supporting  judgments,  evidence,  and  uncertainties  to  others. 

2.5  Background 

Before  we  discuss  the  specifics  of  the  FAC,  we’d  like  to  review  some  of  the  conclusions  we’ve 
reached  based  on  our  research.  These  results  provide  the  foundation  for  the  approaches  we 
recommend  in  the  FAC. 
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2.5.1.  Analysis  Processes 

What  is  the  process  used  to  accomplish  analysis?  Perhaps  the  first  explicit  statement  on  an 
intelligence  analysis  process  was  provided  by  Sherman  Kent.5  Kent’s  process  included  the 
following  steps: 


1 .  The  appearance  of  a  problem  requiring  the  attention  of  a  strategic  intelligence  staff. 

2.  Analysis  of  this  problem  to  discover  which  facets  of  it  are  of  actual  importance  to  the 
U.S.  and  which  of  several  lines  of  approach  are  most  likely  to  be  useful  to  its 
governmental  customers. 

3.  Collection  of  data  bearing  upon  the  problem  as  formulated  in  stage  2.  This  involves  a 
survey  of  data  already  at  hand  and  available  in  the  libraries  of  documentary  materials, 
and  an  endeavor  to  procure  new  data  to  fill  in  gaps. 

4.  Critical  evaluation  of  the  data  thus  assembled. 

5.  Study  of  the  evaluated  data  with  the  intent  of  finding  some  inherent  meaning.  The 
moment  of  the  discovery  of  such  a  meaning  can  be  called  the  moment  of  hypotheses. 

In  reality  there  is  rarely  such  a  thing  as  one  moment  of  hypothesis  though  some 
students  of  method,  largely  as  a  convenience,  speak  as  if  there  were.  Nor  can  it  be 
said  categorically  at  what  stage  in  the  process  hypotheses  appear.  One  would  be 
pleased  to  think  that  they  appeared  at  this,  the  respectable  stage  5,  but  in  actual 
practice  they  begin  appearing  when  the  first  datum  is  collected.  They  have  been 
known  to  appear  even  before  that,  and  they  may  continue  to  appear  until  the  project  is 
closed  out — or  even  after  that,  [emphasis  added] 

6.  More  collecting  of  data  along  the  lines  indicated  by  the  more  promising  hypotheses,  to 
confirm  or  deny  them. 

7.  Establishment  of  one  or  more  hypotheses  as  truer  than  others  and  statement  of  these 
hypotheses  as  the  best  present  approximation  of  truth.  This  stage  is  often  referred  to  as 
the  presentation  state  [sic]. 

While  we  don’t  have  data  on  what  analysts  actually  use,  in  the  literature  Kent’s  description  of  the 
analytical  process  seems  to  have  been  universally  accepted  until  1980.  At  the  time,  Heuer  put 
forward  a  different  description  of  the  intelligence  analysis  process.6  Heuer  described  the 
intelligence  analysis  process  as: 

•  Definition  of  the  analytical  problem 

•  Preliminary  hypotheses  generation 

•  Selective  data  acquisition 

•  Refinement  of  the  hypotheses 

•  Additional  data  collection 

•  Data  interpretation  and  evaluation 

•  Hypotheses  selection 

•  Continued  monitoring 


5 

6 


Kent,  S.  Strategic  Intelligence  for  American  World  Policy,  Princeton  University  Press,  Princeton,  NJ,  1949. 
Heuer,  R.  "Strategies  for  Analytical  Judgment,"  Studies  in  Intelligence,  Vol.  25,  No.  2,  1981,  pp.  66-67. 
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While  both  Kent’s  and  Heuer’s  descriptions  have  much  in  common,  a  fundamental  difference 
exists  in  the  cognitive  process  used  to  get  from  an  understanding  of  the  question  being  asked  to 
the  analytical  answers  for  the  questions  asked.  The  Kent  approach  can  be  characterized  as  a 
bottom  up  or  evidence  in  search  of  a  hypothesis  or  hypotheses.  In  contrast  the  Heuer  approach 
can  be  described  as  a  top  down  or  hypotheses  in  search  of  evidence. 

The  net  effect  is  that  in  the  Kent  approach  information  is  sought  and  meaning  assigned  based  on 
the  relevance  to  the  questions  being  addressed  after  a  thorough  exploration  of  the  issue  or 
question.  In  contrast,  in  the  Heuer  approach  information  is  sought  and  evaluated  based  on  a  set 
of  proposed  answers  to  the  question  being  addressed.  In  both  approaches,  possible  answers  and 
often  ultimately  a  single  answer  to  the  question  must  be  proposed  and  selected  as  the  answer  to 
the  question  under  study. 

The  core  difference  is  the  means  and  basis  for  generating  the  answer(s).  In  the  one  instance,  the 
hypotheses  come  from  analysis  of  all  relevant  information  to  the  question.  In  the  other  instance, 
the  hypotheses  are  generated  based  the  inherent  knowledge  of  the  analysts  and  before  the 
relevant  infonnation  is  acquired.  This  may  not  appear  to  be  a  difference  that  can  affect  the 
quality  of  the  answer  to  the  issue  being  addressed,  but  it  is.  The  contextual  framework  in  which 
meaning  is  assigned  to  information  is  different  between  the  two  approaches.  More  will  be 
explained  in  subsequent  portions  of  the  FAC. 

Both  Heuer’s  and  Kent’s  descriptions  of  the  analysis  process  continue  to  be  used  by  those  who 
write  on  intelligence  analysis.  In  most  cases  they  write  without  the  acknowledgement  of  the 
existence  of  the  other,  which  is  an  importantly  different  process.  We  know  of  no  effort  to 
evaluate  the  two  processes  relative  to  one  another  or  to  identity  the  circumstances  where  one 
process  is  more  suitable  than  the  other;  nor  is  there  data  to  indicate  the  relative  use  of  each  of  the 
processes  by  analysts.  However,  in  the  known  training  courses  for  analysts,  preference  is  given 
to  the  process  as  described  by  Heuer. 

To  develop  the  content  of  the  FAC  and  provide  a  coherent  guide  for  your  analysis  efforts,  a 
choice  has  to  be  made  between  the  process  as  described  by  Kent  and  the  one  described  by  Heuer. 
Based  on  our  evaluation  of  the  literature  on  intelligence  analysis  and  the  findings  in  the  scientific 
research,  we  have  chosen  to  use  the  process  as  described  by  Kent  as  the  underlying  foundation. 
We  will  use  this  overall  description,  with  significant  elaboration,  to  build  a  holistic  framework 
for  doing  intelligence  analysis,  which  you  can  use  to  guide  your  efforts  and  stimulate  your 
thinking  as  you  strive  to  do  high  quality  analysis  that  is  useful  for  your  customers. 

As  we’ll  describe  throughout,  there  is  a  scientific  basis  for  all  of  the  elements  described  by  each 
of  the  FAC  sections  and  sound  reasons  for  doing  them  all  as  explained.  While  these  elements  are 
approached  in  a  specific  order  because  of  the  natural  flow  of  the  work  and  for  purposes  of 
explanation,  as  you  read  them  and  do  your  analysis,  you’ll  find  that  they  are  not  independent 
linear  steps,  but  very  interdependent  pieces  of  a  whole,  consideration  of  which  must  shape 
each  individual  activity  you  undertake.  As  we’ll  explain,  you  will  also  find  that  you’ll  cycle 
back  through  many  of  the  elements  repeatedly  until  you’re  satisfied  with  the  sufficiency  of  what 
can  be  done  for  each,  given  a  variety  of  constraints  (e.g.,  information,  time,  etc). 

2.5.2.  Analysis  Overview 

As  mentioned  in  the  Introduction,  an  analyst  has  the  responsibility  to  acquire  knowledge  and 
produce  assessments  of  foreign  activities,  capabilities,  intentions,  and  plans  in  his  or  her  area  of 
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assignment.  The  decision  making  process  of  an  analyst’s  target  of  study  (e.g.,  a  foreign  entity  or 
country)  will  affect  the  target’s  activities,  capabilities,  intentions,  and  plans.  This  decision 
making  process  depends  on  consideration  of  many  factors  and  conditions  that  are  outside  of  an 
individual  analyst’s  area  of  responsibility;  it  is  very  dependent  on  the  target’s  activities, 
capabilities,  intentions,  and  plans  in  areas  that  are  the  responsibilities  of  other  analysts.  Thus  to 
achieve  his  or  her  objective  of  knowing  or  predicting  the  outcome  of  a  target’s  act  of  decision 
making,  each  analyst  requires  and  is  dependent  upon  the  knowledge  and  assessments  of  other 
analysts. 

An  excellent  example  of  this  interdependency  is  provided  in  Kent’s  discussion  of  the  assessment 
of  the  strategic  capabilities  of  Great  Frusina.7  The  incorporation  of  the  knowledge,  judgments, 
or  assessments  provided  by  others  is  solely  the  task  of  an  individual  analyst  in  the  formulation  of 
judgments  within  his  or  her  area  of  responsibility.  Other  analysts  and  supervisors  may  comment 
and  disagree  with  the  judgments,  but  their  involvement  does  not  alter  the  fact  that  a  specific 
individual  originates  the  judgments  related  to  a  specific  area,  and  that  judgment  is  an  inherent 
responsibility  of  the  analyst  assigned  to  that  area.  As  a  result  of  this  division  of  responsibility, 
analysis  is  a  group  activity  in  the  context  of  inputs  from  multiple  participants  while  at  the  same 
time  the  results  are  the  activity  of  a  sole  individual. 

The  fact  that  the  very  cognitive  aspects  of  the  analysis  process  are  the  responsibility  of  an 
individual  has  major  ramifications  on  the  outcome  of  the  analysis  process.  Universal  agreement 
exist  that  analysis  is  a  highly  cognitive  process.  The  analyst  is  subjected  to  a  large  volume  of 
information  or  data  which  he  or  she  must  process  in  order  to  acquire  knowledge  and  make 
judgments. 

How  an  individual  processes  information,  incorporates  prior  information  or  knowledge,  and 
draws  conclusions  from  the  information  has  been  the  subject  of  a  substantial  amount  of  research 
Cognitive  scientists  have  yet  to  agree,  except  in  broad  descriptions  of  the  process,  but  agreement 
does  exist  that  the  outcome  of  the  process  is  unique  to  an  individual.  As  a  consequence,  the 
judgments  resulting  from  the  processing  of  information  can  differ  from  individual  to  individual, 
or  might  be  the  same  but  for  very  different  reasons.  This  variability  between  individuals  affects 
not  only  the  outcome  but  also  the  very  core  of  the  analysis  process:  the  selection  of  information 
that  is  relevant  to  the  intelligence  issues  being  addressed  and  the  assignment  of  meaning  to  the 
selected  information. 

2.5.3.  Information  and  the  Individual 

The  intelligence  and  scientific  literatures  have  numerous  statements  on  the  interdependency 
between  information  and  the  individual,  as  manifested  in  the  following: 

Far  more  than  most  of  us  realize,  the  answers  we  get  in  our  thinking  are  influenced  by 
subjective  factors — factors  residing  in  the  subject  who  does  the  viewing  rather  than  in 
the  object  viewed.  Our  value  judgments  detennine  which  objects  and  activities  we  seek 
and  which  we  avoid.  Habits  and  knowledge  acquired  from  past  experience  determine  the 
methods  by  which  we  satisfy  our  needs.  The  needs,  value  judgments,  and  past 
experience  of  the  individual  form  a  complex  point  of  view  from  which  he  sees  all  new 
experiences  8 


7  Kent  pp.  40-65. 

s  Little,  W.W.  Applied  Logic,  Houghton  Mifflin,  Boston,  MA,  1955,  p.  261. 
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Yet,  in  practice  the  following  seems  to  be  the  more  prevalent  view: 


There  is  a  widespread  myth  that  information  is  something  in  the  world  that  does  not 
depend  on  the  point  of  view  of  the  observers  and  that  it  is  (or  is  often)  independent  of  the 
context  in  which  it  occurs.9 

Woods  et  al.  also  pointed  out  the  fallacy  of  this  myth:  This  is  simply  not  the  case.  There 
are  no  facts  of  fixed  significance.  The  available  data  are  raw  materials.  A  particular 
datum  gains  significance  or  meaning  only  from  its  relationship  to  the  context  in  which  it 
occurs  or  could  occur  including  the  perspective  of  observers.  As  a  result, 
informativeness  is  not  a  property  of  the  data  field  alone,  but  is  a  relationship  between 
observers  and  the  data  field.10 

The  basis  for  the  often  claimed,  “facts  don’t  speak  for  themselves”  is  implicit  in  these 
statements.  The  context  needed  for  significance  and  meaning  requires  an  input  from  the  analyst 
of  the  infonnation.  What  is  this  input?  This  topic  will  be  covered  in  more  detail  later  in  the 
FAC,  but  suffice  it  to  say  that  the  input  includes  the  knowledge,  beliefs,  values,  and  ideas  of  the 
analyst  of  the  information,  as  well  as  the  purpose  for  reviewing  the  infonnation.  In  other  words 
the  input  is  based  on  the  total  content  on  the  mind  of  the  analyst.  Since  the  content  of  each 
person’s  mind  is  different  than  all  others,  the  fact  that  the  interpretation  of  information  is 
different  from  analyst  to  analyst  is  hardly  surprising. 

Another  factor  that  must  be  considered  in  this  relationship  between  the  infonnation  and  the 
individual  is  the  fact  that  the  content  of  mind  is  not  static.  The  purpose  for  interpreting  the 
information  can  change  when  new  intelligence  issues  arise.  Information  that  was  worthless 
when  first  examined  can  be  very  valuable  given  a  new  issue  to  answer.  Also,  while  some  values 
and  beliefs  are  slow  to  change,  an  individual  is  constantly  adding  new  knowledge,  particularly 
related  to  the  accomplishment  of  a  task  such  as  answering  an  intelligence  issue.  Thus  the 
interpretation  of  information  being  analyzed  can  change  as  the  mind  changes.  What  may  have 
been  insignificant  yesterday  may  be  very  important  today.  The  implication  is  a  need  to 
continuously  reinterpret  the  information. 

2.5.4.  Acquisition  of  Relevant  Information 

The  analyst  must  have  a  full  and  comprehensive  understanding  of  the  intelligence  issue  to  be 
addressed  before  any  other  worthwhile  activities  can  be  accomplished.  Once  there  is 
understanding  of  the  intelligence  issue,  the  acquisition  of  information  that  is  relevant  to  the  issue 
can  begin. 

What  is  relevant  information?  Merrill-Webster  defines  relevant  as: 

•  having  significant  and  demonstrable  bearing  on  the  matter  at  hand,  or 

•  affording  evidence  tending  to  prove  or  disprove  the  matter  at  issue  under  discussion 


9Woods,  D.D.,  Patterson,  E.S.  &  Roth,  E.M.  Can  we  Ever  Escape  from  Data  Overload?  A  Cognitive  Systems 
Diagnosis,  AFRL-HE-WP-TR- 1999-0 195,  Ohio  State  University  and  Roth  Cognitive  Engineering,  Columbus,  OH, 
1998,  p.  14. 

10  Woods  p.  14. 
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However,  as  you’ll  see  in  subsequent  sections,  in  the  FAC  relevance  is  defined  in  a  very 
straightforward  way:  The  information  that  you  find  is  relevant  if  it  pertains  to  your  intelligence 
issue. 

Relevance  will  be  discussed  much  more  in  the  FAC,  but  for  now  it  is  important  to  point  out  that 
the  question  of  what  is  relevant  information  is  often  viewed  as  a  minor  problem.  Often  the  view 
is  that  all  information  is  what  it  is,  with  a  single  meaning,  but  research  indicates  this  view  is 
wrong.  An  extensive  amount  of  research  has  been  performed  on  subject  matter  experts  judging 
the  relevance  of  information  for  a  specific  information  need.  The  result  was  little  agreement  on 
which  information  was  relevant.  These  results  were  unexpected  at  the  time  (about  40  years  ago), 
but  based  on  the  factors  described  in  the  prior  section,  the  results  are  easily  explained. 

Numerous  investigations  have  been  subsequently  done  and  the  results  are  the  same;  different 
people  have  substantial  disagreement  on  what  information  is  relevant  to  a  specified  need. 1 1 
(Schamber  1994) 

Relevance  is  determined  by  an  interaction  between  the  information  and  the  user  of  that 
information.  The  existence  of  this  dependency  is  an  explanation  for  performance  of  information 
storage  and  retrieval  (IS&R)  systems.  The  logic  used  in  these  systems  to  identify  relevant 
information  is  based  on  the  words  used  to  express  the  content  of  the  information  system.  To 
date,  none  have  been  able  to  incorporate  the  content  of  the  mind  of  the  user,  or  the  interaction 
between  the  user  and  the  information,  necessary  for  making  judgments  on  the  relevance  of  the 
information. 

To  overcome  this  inherent  limitation  of  the  capabilities  of  IS&R  systems,  the  user  must  develop 
words  and  phrases  that  might  be  contained  in  the  information  being  sought.  The  selection  of 
which  of  the  words  and  phrases  to  use  requires  an  information  seeking  tradeoff.  One  can  search 
using  multiple  words  and  phrases  in  an  attempt  to  retrieve  all  the  relevant  information;  but  as  a 
consequence,  the  IS&R  system  will  provide  a  large  amount  of  non-relevant  information.  Or,  one 
can  search  with  a  very  limited  set  of  words  and  phrases  in  order  to  minimize  the  amount  of  non- 
relevant  information  provided,  but  by  doing  so  will  not  retrieve  much  of  the  relevant 
information. 

The  preceding  tradeoff  poses  a  major  operational  dilemma  for  the  analyst.  The  pursuit  of  all 
relevant  information  can  consume  a  large  amount  of  time  with  no  foreknowledge  that  the  results 
will  be  worth  the  effort.  However,  an  inadequate  and  incomplete  base  of  relevant  information 
can  preclude  the  assessment  of  a  valid  and  complete  answer  to  the  intelligence  issue  or  question. 

2.5.5.  Extracting  Meaning  from  Information 

After  the  acquisition  of  relevant  information,  attributing  meaning  to  the  information  can  begin. 
Many  people  have  described  the  essence  of  the  analysis  process  as  assigning  meaning  to  the 
information,  but  few  have  been  explicit  about  what  they  mean  by  “meaning.”  A  2008  dictionary 
definition  is: 

•  “significant  quality:  especially:  implication  of  a  hidden  or  special  significance 
(Merriam-Webster  Online  Dictionary) 


1 1  Schamber,  L.  "Relevance  and  Information  Behavior,"  Annual  Review  of  Information  Science  and  Technology’, 
Vol.  29,1994,  pp.  3-48 
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This  definition  is  ideal  for  intelligence  analysis  since  the  objective  of  analysis  is  to  reveal  the 
hidden  information  and  to  determine  that  which  someone  doesn’t  want  you  to  know.  Unless 
analysis  results  in  something  that  was  not  explicitly  contained  in  the  information  under 
analysis,  analysis  has  not  been  done;  instead,  the  result  is  reporting. 

From  an  overall  perspective,  the  meaning  attributed  to  infonnation  falls  in  two  categories:  what 
has  been  accomplished  and  why.  Most  analysis,  intelligence  or  other,  deals  with  the  former 
(what).  But,  the  latter  (why)  is  more  likely  to  be  an  indicator  of  the  future.  Primary  attention 
must  be  given  to  the  substantive  content  of  infonnation  to  determine  changes  in  one’s  state  of 
knowledge  and/or  situational  awareness.  Familiarity  with  the  substantive  content  is  the  requisite 
knowledge  for  making  a  determination  of  changes  in  the  what.  Subject  matter  experts  have  this 
knowledge  so  they  can  be  effectively  used  to  review  intelligence  infonnation  and  answer  the 
question  of  what  has  been  accomplished.  However  while  subject  matter  expertise  is  essential, 
the  detennination  of  why  requires  much  more  knowledge  about  attributes  other  than  the  current 
subject  matter  (i.e.,  prior  aspects  of  substantive  content,  human  motivations,  relevance  and  value 
relative  to  the  issue)  than  most  subject  matter  experts  outside  the  intelligence  organizations 
possess,  as  will  be  explained  in  the  next  paragraph.  If  others  outside  of  intelligence 
organizations  do  attribute  the  why,  they  will  most  likely  be  practicing  “mirror  imaging”  based  on 
their  own  experience,  not  the  particulars  of  the  substantive  content,  human  motivations,  and 
relevance  and  value  to  the  issue  at  hand.  In  other  words,  they  will  assign  to  others  our 
motivations  for  similar  activities. 

Attributing  the  why  is  much  more  complex  than  solely  understanding  the  subject  matter  or 
substantive  content  and  the  what.  Based  on  our  research,  we  have  detennined  that  all  of  the 
following  must  be  ascertained  in  the  context  of  information  that  relates  to  the  actions,  behavior, 
and  decisions  of  humans: 

•  Substantive  content 

•  Motivation  of  the  human  actors 

•  Relevance  of  the  information  to  the  issue  under  study 

•  Value  of  infonnation  to  the  issue  under  study 

Answers  to  the  questions  associated  with  each  of  these  four  attributes  must  be  established  for 
every  item  of  information  reviewed  by  the  analysts.  Some  answers  will  be  established  easily,  but 
others  will  require  a  considerable  amount  of  research  and  added  information.  The  following  are 
examples  of  the  questions  and  additional  infonnation  needed  to  determine  the  answers  for  the 
four  preceding  attributes: 

•  Substantive  content.  Has  the  person/organization/country  engaged  in  such 
substantive  activities  prior  to  the  new  information?  If  so,  how  does  this  activity 
differ  from  the  past  activities?  How  does  the  substantive  content  relate  to  the  issue 
under  analysis? 

•  Motivation  of  human  actors.  If  individual(s)/organization(s)/countries  are 
associated  with  the  information,  have  they  previously  been  associated  with  similar 
activities?  If  so,  how  does  the  new  information  differ  from  past  activities  by  those 
individuals(s)/organization(s)/countries?  If  not,  how  does  this  new  activity  relate  to 
past  activities/responsibilities  of  those  individual(s)/  organization(s)/countries? 
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•  Relevance  of  information.  If  previously  determined  that  the  substantive  content 
does  not  relate  to  issue  under  study,  the  information  may  not  be  relevant.  However, 
if  the  individual(s)/organization(s)/countries  associated  with  the  information  are  of 
interest  to  the  issue  under  study,  the  information  is  relevant  regardless  of 
substantive  content. 

•  Value  of  information.  If  the  answers  to  the  above  questions  change  the  analyst’s 
knowledge  and  understanding  on  any  aspect,  then  the  information  is  valuable.  If 
the  answers  only  confirm  what  was  previously  known,  the  information  is  of  limited 
value.  If  no  change  occurs  in  the  analyst’s  knowledge,  the  information  has  no  value 
to  the  issue  under  study. 

Based  on  the  preceding  discussion,  the  meaning  of  information  can  hardly  ever  occur  based 
solely  on  the  content  of  the  information  being  assessed.  The  assignment  of  meaning  requires 
extensive  prior  knowledge  about  substantive  matters,  individuals,  organizations  and  nations  as 
well  as  the  assignment  of  meaning  to  the  prior  information.  If  the  analyst  does  not  have  this 
knowledge,  he  or  she  cannot  fully  assign  meaning  to  the  current  information. 

2.6  Summary 

We  conclude  this  section  of  the  FAC  with  the  following  observations: 

•  An  analyst  is  responsible  for  the  production  of  assessments  of  foreign  capabilities, 
intentions,  and  plans  in  his  or  her  area  of  assignment. 

•  The  assignment  of  meaning,  revealing  hidden  implications,  or  identification  of 
significance  to  individual  pieces  or  aggregates  of  information  is  the  essence  of 
intelligence  analysis. 

•  Information  does  not  have  single  meaning.  Its  meaning  is  dependent  upon  the 
individual  making  the  assignment  of  meaning. 

•  The  individual  characteristics  that  affect  the  assignment  of  meaning  include 
knowledge,  beliefs,  values,  ideas,  and  the  purpose  for  assigning  the  meaning. 

•  The  meaning  of  information  ideally  requires  knowledge  of  all  prior  information 
related  to  the  current  information. 
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3.0 

3.1 


ANALYSTS  AS  INDIVIDUALS 
Introduction 


In  the  “Introduction  to  the  FAC”  section,  we  tried  to  convey  the  message  that  you,  as  an 
individual  doing  intelligence  analysis,  have  a  singular  role  of  paramount  importance.  The 
formulation  of  judgments  about  foreign  activities  and  intentions  related  to  your  assigned  area  is 
your  responsibility  and  only  yours.  These  judgments  are  the  foundation  for  all  subsequent 
deliberation  by  peers  and  superiors  as  they  form  judgments  about  other  areas  of  foreign  activities 
that  are  related  to  your  area  of  responsibility. 

The  formulation  of  these  judgments  is  based  on  your  assignment  of  meaning  to  the  all  relevant 
information  available  to  you.  As  pointed  out  earlier,  the  assignment  of  relevancy  and  meaning  is 
dependent  on  the  content  of  the  information,  the  intelligence  issue  being  addressed,  and  the 
totality  of  the  content  of  your  mind.  These  are  the  inputs  to  your  cognition.  Your  cognitive 
ability  and  its  application  detennine  how  these  inputs  will  be  utilized  to  form  judgments. 

One  important  observation  from  the  literature  on  cognition  is  that  there  is  no  threshold  of 
cognitive  ability  or  content  of  the  mind  for  making  decisions.  The  quality  of  the  information 
inputs  or  the  degree  of  cognitive  ability  does  not  limit  the  formation  of  judgments.  People  will 
make  judgments  on  the  meaning  of  information  regardless  of  what  they  know  or  how  they  think. 
Thus  if  you  wish  to  improve  the  quality  of  your  judgments,  as  distinct  from  the  act  of  making 
judgments,  one  must  develop  a  realistic  appraisal  of  your  cognitive  abilities  and  the  content  of 
your  mind.  To  do  this,  however,  means  that  one  has  to  know  the  specifics  of  what  has  to  be 
appraised.  For  those  with  this  inclination,  the  intent  of  this  section  is  to  provide  assistance.  This 
section  deals  with  the  mechanics  of  thinking,  inputs  to  cognition,  and  the  outputs  from  various 
forms  of  cognition.  We  also  provide  a  description  of  how  writing  can  enhance  cognition. 

3.2  Thinking  about  Thinking 

The  importance  of  the  individual  in  analysis  is  explicitly  recognized  in  the  extensive  literature  on 
intelligence  analysis  by  acknowledging  the  analyst’s  contribution  to  the  intelligence  product.  At 
the  same  time  when  deficiencies  occur,  as  represented  by  intelligence  failure,  these  deficiencies 

are  often  attributed  to  faulty  thinking  on  the  part  of  the  analyst.  The  importance  of  how  one 
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thinks  was  emphasized  by  Davis: 

Effective  management  of  the  impact  of  cognitive  biases  and  other  psychological 
challenges  to  the  analytic  process  is  at  least  as  important  in  ensuring  the  soundness  of 
assessments  on  complex  issues  as  the  degree  of  substantive  expertise  invested  in  the 
effort. 

As  might  be  expected  then,  most  of  the  recommendations  for  improvement  relate  to  the  analyst 
doing  better  thinking.  Almost  universally,  the  recommendations  are  for  analysts  to  be  rational 
and  unbiased.  These  recommendations  are  usually  silent  on  how  this  is  to  be  accomplished 
except  for  admonishments  to  use  critical  thinking,  which  is  frequently  not  defined  or  defined  in 
numerous  ways.  In  effect,  these  recommendations  imply  that  desired  end  state  of  analysis  is  easy 
to  achieve.  It  isn’t,  as  manifested  by  the  centuries  old  debate  among  psychologists  and 
philosophers  on  the  topics  of  rationality  and  bias. 


12  Davis,  J.  "Why  Bad  Things  Happen  to  Good  Analysts,"  in  Analyzing  Intelligence:  Origins,  Obstacles,  and 
Innovations,  eds.  R.Z.  George  &  J.B.  Bmce,  2008,  p.  157. 

16 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 
88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


Nevertheless,  recent  developments  in  this  debate  do  provide  an  understanding  of  the  factors  that 
lead  to  irrationality  and  bias.  To  provide  you  insight  into  these  developments  and  their 
applicability  to  your  evolution  as  an  analyst,  we  will  cover  infonnation  on  the  functional 
partitions  of  the  mind,  the  role  of  each  partition,  the  relationships  and  dependencies  of  each 
partition,  the  inputs  to  the  functioning  of  each  partition,  and  thinking  styles.  Using  what  is  now 
known  about  the  mind,  albeit  with  considerable  effort,  can  result  in  analysis  that  has  the  desired 
characteristics  of  rationality.  The  intent  of  this  FAC  section  is  to  provide  basic  knowledge  to 
facilitate  you  achieving  the  holy  grail  of  analysis:  being  rational  and  unbiased. 

So  you  are  challenged  to  be  unbiased.  What  does  this  mean?  The  answer  may  not  be  as  obvious 
as  you  think.  The  Merriam-Webster  online  dictionary  defines  bias  as  “an  inclination  of 
temperament  or  outlook;  especially,  a  personal  and  sometimes  unreasoned  judgment”.  Wikipedia 
says  “the  term  biased  is  used  to  describe  an  action,  judgment,  or  other  outcome  influenced  by  a 
prejudged  perspective.  It  is  also  used  to  refer  to  a  person  or  body  of  people  whose  actions  or 
judgments  exhibit  bias.”  We  think  you  would  agree  that  this  is  the  meaning  most  often  assigned 
in  common  usage  when  bias  is  discussed  relative  to  gender,  culture,  race,  nationality,  and 
religion.  However,  this  is  not  the  meaning  of  the  word  biased  as  used  in  the  intelligence 
literature  nor  is  it  what  is  meant  when  they  call  for  analysis  to  be  unbiased,  although  they 
certainly  don’t  want  the  analytical  results  based  on  stereotypes.  Rather,  the  intelligence  literature 
is  talking  about  cognitive  biases.  In  the  psychological  literature,  cognitive  bias  is  thinking  that 
deviates  from  normative  assumptions.  What  are  normative  assumptions? 

Basically,  an  assumption  about  “how  things  should  be”  is  a  normative  assumption, 
because  it  presupposes  the  existence  of  a  “normal  example”  that  all  things  should 
follow.13 

Heuer14  provides  another  view  on  cognitive  biases. 

A  substantial  body  of  research  in  cognitive  psychology  and  decision-making  is  based  on 
the  premise  that  these  cognitive  limitations  cause  people  to  employ  various  simplifying 
strategies  and  rules  of  thumb  to  ease  the  burden  of  mentally  processing  information  to 

make  judgments  and  decisions . Cognitive  biases  are  mental  errors  caused  by  our 

simplified  information  processing  strategies. 

Please  make  a  mental  note  of  the  first  sentence  with  the  implications  that  cognitive  biases  are 
causes  by  defects  in  the  mind.  We’ll  return  to  this  topic  later. 

In  both  usages,  the  word  bias  is  describing  irrational  behavior.  However,  the  source  for  this 
irrationality  is  different  for  cognitive  biases  and  stereotypical  biases.  In  neither  case  is  the 
irrationality  due  to  some  inherent  cognitive  limitation  as  stated  by  Heuer.  By  understanding  the 
source  of  each  form  of  irrationality,  corrective  action  can  be  taken  so  rational  thought  is  more 
likely  to  occur,  as  we’ll  explain  later. 

The  inference  from  the  preceding  is  that  irrational  thinking  will  be  eliminated  if  only  we  can 
control  our  cognitive  biases.  This  is  an  incorrect  inference  since  irrationality  originates  from 
numerous  sources.  The  situation  becomes  even  more  complicated  by  the  fact  that,  except  in  the 
use  of  logic,  what  is  rational  is  subject  to  interpretation  since  the  standards  forjudging  rationality 


13  Some  definitions  of  terms,  19  October  1997,  http://www.bcholmes.org/defmitions.html. 

14  Heuer,  R.J.J.  Psychology’  of  Intelligence  Analysis,  Center  for  the  Study  of  Intelligence,  Washington,  D.  C.,  1999, 
p.  93. 
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are  often  established  by  the  individual.  An  example  of  this  is  the  above  definition  of  normative 
assumptions.  Rational  thinking  is  what  most  people  judge  to  be  rational.  In  effect,  rational 
thinking  is  what  best  conforms  to  evidence  and  accepted  beliefs.  This  state  of  affairs  puts  the 
burden  of  proving  that  your  analysis  is  rational  thinking  on  you.  Your  challenge  is  to 
appropriately  use  evidence  and  to  know  and  defend  your  beliefs.  Incidentally,  rational  thinking 
does  not  necessarily  equate  with  truth  as  demonstrated  by  Sherman  Kent’s  estimate  on  the 
Soviets  placing  missiles  in  Cuba.  In  this  case,  there  was  a  careful  evaluation  of  the  available 
evidence  and  open-minded,  thoughtful  consideration  of  intentions  from  the  perspective  of  the 
Soviets.  However,  “truth”  didn’t  follow  a  rational  path  in  this  situation  where  the  estimate  was 
made  before  the  enemy  made  their  decision.  As  Kent  indicated  in  his  post-analysis,  they  did  not 
“divine  exactly  when  the  enemy  is  about  to  make  a  dramatically  wrong  decision.”15 

3.2.1,  Background 

Since  at  least  the  days  of  the  Greek  philosophers,  the  state  of  mind  has  been  a  topic  of  great 
discussion.  Plato  felt  that  the  mind  was  divided  into  two  separate  components  of  reason  and 
emotion  that  work  at  odds  with  one  another.  While  Plato’s  concept  of  a  divided  mind  did  not 
totally  disappear  from  consideration,  the  dominant  theme  that  has  evolved  since  the  Middle  Ages 
was  based  on  the  work  of  Descartes,  who  agreed  in  principal  with  Plato  but  gave  more  emphasis 
on  the  ability  to  reason.  His  faith  in  rationality  was  accepted  and  became  the  foundation  for 
thinking  about  the  human  character  for  the  next  400  years.  The  concept  that  man  is  rational  and 
would  make  decisions  that  served  his  best  interests  became  the  bedrock  for  describing  human 
behavior,  particularly  in  such  fields  as  economics.  However,  research  was  increasingly  showing 
that  man  frequently  did  not  exhibit  rational  behavior,  but  the  research  is  not  without  controversy. 

Thirty  years  of  decision  research  has  used  rational  theories  from  economics,  statistics, 
and  logic  to  argue  that  descriptive  behavior  falls  systematically  short  of  normative  ideals. 
But  this  apparent  gap  between  the  normative  and  the  descriptive  has  provoked  many 
debates:  Is  there  in  fact  a  gap?.. Many  economists  and  philosophers  have  argued  on 
principle  that  there  is  no  gap:  people  are  essentially  rational,  any  errors  are  random  and 
non-systematic,  and  apparent  systematic  discrepancies  are  attributable  to  improper 
empirical  methods.16 

The  so-called  biases  are  mostly  found  in  laboratory  studies  using  artificial  puzzle  tasks 
and  college  freshmen  as  subjects,  conditions  that  minimize  expertise  and  context.  In 
natural  settings,  biases  can  disappear  or  be  greatly  reduced.17 

The  counter  arguments  are: 

They  (Kahneman  and  Tversky)  chipped  away  at  it  by  carefully  documenting  how  people 
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violated  one  bedrock  nonnative  assumption  after  another. 

The  existence  of  systematic  biases  is  now  largely  accepted  by  decision  researchers,  and, 
increasingly,  by  researchers  in  other  disciplines.19 

15  Kent,  S.  &  Steury,  D.P.  Sherman  Kent  and  the  Board  of  National  Estimates:  Collected  Essays,  Center  for  the 
Study  of  Intelligence,  Washington,  D.C.,  1994 

16  Larrick,  R.P.  "Debiasing,"  in  Blackwell  E[andbook  of  Judgment  and  Decision  Making,  eds.  D.J.  Koehler  &  N. 
Harvey,  Blackwell,  Malden,  MA,  2004,  p.  316. 

17  Klein,  G.,  Moon,  B.  &  Hoffman,  R.R.  "Making  Sense  of  Sensemaking  1:  Alternative  Perspectives,"  IEEE 
Intelligent  Systems,  Vol.  21,  No.  4,  2006,  p.  72. 

18  Tetlock,  P.E.  &  Mellers,  B.A.  "The  Great  Rationality  Debate,"  Psychological  Science,  Vol.  13,  No.  1,  2002,  p.  94. 
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A  substantial  research  literature  -  one  comprising  literally  hundreds  of  empirical  studies 

conducted  over  nearly  three  decades  -  has  firmly  established  that  people’s  responses 

20 

often  deviate  from  the  perfonnance  considered  nonnative  on  many  reasoning  tasks. “ 

This  debate  is  not  likely  to  end  soon  since  the  probable  cause  of  the  disagreement  is  not  the 
interpretation  of  the  experimental  data,  but  rather  the  nature  of  the  experiments  (e.g.,  the 
circumstances  and  context,  the  methods  and  type  of  data  collected,  etc).  Disagreement  does  not 
exist  over  the  fact  that  people  will  exhibit  cognitive  biases  in  many  different  circumstances.  For 
our  purposes,  this  disagreement  is  not  significant  other  than  you  should  be  aware  of  it.  As 
mentioned  above,  in  the  appropriate  setting  the  biases  can  be  eliminated  or  largely  reduced.  Our 
intent  is  to  provide  the  requisite  information  so  one  can  eliminate  or  largely  reduce  their  biases. 
We  will  return  to  what  is  required  to  achieve  this  mitigating  effect  on  cognitive  biases  later. 

During  the  last  50  years,  marked  changes  in  thinking  have  occurred  about  how  humans  think. 
Darwinism  and  the  idea  that  the  mind  is  a  product  of  evolution  have  had  an  increasing  influence 
on  the  consideration  of  the  brain  and  the  mind.  Increasingly,  Plato’s  view  of  a  competition 
between  rationality  and  emotion  has  become  the  foundation  for  concepts  about  the  functioning  of 
the  mind.  Among  other  concepts,  this  has  led  to  the  two  mind  concept  that  is  continuing  to 
evolve  as  an  explanation  for  human  cognitive  behavior. 

Various  theories  of  the  two  mind  concept  have  been  put  forth.  It  is  beyond  the  scope  of  this 

2  1 

effort  to  judge  which  of  these  theories  is  most  valid.  However,  the  work  of  Keith  E.  Stanovich” 
is  the  most  useful  for  our  purpose;  that  is,  using  the  evolving  knowledge  about  how  one  thinks  to 
guide  what  to  do  to  achieve  rational  and  unbiased  analysis.  Stanovich’s  description  of  how  the 
mind  works  follows,  as  well  as  some  descriptions  by  others. 

3.2.2.  How  the  Mind  Functions 

Stanovich  says  that  general  agreement  exists  that  two  different  types  of  cognitive  process  occur 
in  the  brain:  Type  1  and  Type  2.  These  processes  are  further  defined  as: 

The  defining  feature  of  Type  1  processing  is  its  autonomy.  Type  1  processes  are  termed 
autonomous  because:  1)  their  execution  is  rapid,  2)  their  execution  is  mandatory  when  the 
triggering  stimuli  are  encountered,  3)  they  do  not  put  a  heavy  load  on  central  processing 
capacity  (that  is,  they  do  not  require  conscious  attention),  4)  they  are  not  dependent  on 
input  from  high-level  control  systems,  and  5)  they  can  operate  in  parallel  without 
interfering  with  each  other  or  with  Type  2  processing.  Type  1  processing  would  include 
behavioral  regulation  by  the  emotions;  the  encapsulated  modules  for  solving  specific 
adaptive  problems  that  have  been  posited  by  evolutionary  psychologists;  processes  of 
implicit  learning;  and  the  automatic  firing  of  overleamed  associations.  Type  1 
processing,  because  of  its  computational  ease,  is  a  common  processing  default.  Type  1 
processes  are  sometimes  termed  the  adaptive  unconscious  in  order  to  emphasize  that 
Type  1  processes  accomplish  a  host  of  useful  things  -  face  recognition,  proprioception, 
language  ambiguity  resolution,  depth  perception,  etc.  -  all  of  which  are  beyond  our 


19  Larrick  p.  316. 

20  Stanovich,  K.E.  &  West,  R.F.  "Individual  Differences  in  Reasoning:  Implications  for  the  Rationality  Debate? 
(Includes  Advancing  the  Rationality  Debate),"  Behavioral  and  Brain  Sciences,  Vol.  23,  No.  5,  2000,  p.  645. 

21  Stanovich,  K.E.  What  Intelligence  Tests  Miss:  The  Psychology  of  Rational  Thought,  Yale  University  Press,  New 
Haven,  CT,  2009. 
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awareness.  Heuristic  processing  is  a  term  often  used  for  Type  1  processing  -  processing 
that  is  fast,  automatic,  and  computationally  inexpensive,  and  that  does  not  engage  in 
extensive  analysis  of  all  the  possibilities. 

Type  2  processing  contrasts  with  Type  1  processing  on  each  of  the  critical  properties  that 
define  the  latter.  Type  2  processing  is  relatively  slow  and  computationally  expensive  -  it 
is  the  focus  of  our  awareness.  Many  Type  1  processes  can  operate  at  once  in  parallel,  but 
only  one  Type  2  thought  or  a  very  few  can  be  executing  at  once  -  Type  2  processing  is 
thus  serial  processing.  Type  2  processing  is  often  language  based  and  rule  based.  It  is 
what  psychologists  call  controlled  processing,  and  it  is  the  type  of  processing  going  on 
when  we  talk  of  things  like  "conscious  problem  solving.22 

He  further  describes  the  relationship  between  the  two  processes  as: 

One  of  the  most  critical  functions  of  Type  2  processing  is  to  override  Type  1  processing. 
This  is  sometimes  necessary  because  Type  1  processing  is  "quick  and  dirty."  This  so- 
called  heuristic  processing  is  designed  to  get  you  into  the  right  ballpark  when  solving  a 
problem  or  making  a  decision,  but  it  is  not  designed  for  the  type  of  fine-grained  analysis 
called  for  in  situations  of  unusual  importance  (financial  decisions,  fairness  judgments, 

23 

employment  decisions,  legal  judgments,  etc.). 

In  effect,  the  identification  of  Type  1  and  Type  2  processes  leads  us  to  back  the  observations 
made  many  centuries  ago  by  Plato.  While  Plato  was  satisfied  with  the  explanation  of  the  mind 
divided  into  two  parts,  Stanovich  postulates  the  existence  of  three  minds:  autonomous, 
algorithmic,  and  reflective.  Type  1  processes  are  accomplished  in  an  autonomous  mind.  Type  2 
processes  that  are  performed  in  both  the  algorithmic  mind  and  the  reflective  mind.  The 
algorithmic  mind,  as  you  might  suspect  from  its  name,  is  associated  with  information  processing 
mechanisms  with  many  of  the  attributes  attributed  to  computers,  including  the  lack  of  emotions. 
In  contrast,  the  reflective  mind  is  “concerned  with  the  goals  of  the  system,  beliefs  relevant  to 
those  goals,  and  the  choice  of  action  that  is  optimal  given  the  system's  goals  and  beliefs.”24 

Stanovich  provides  considerable  detail  on  the  relationship  between  the  three  minds,  outputs  and 
inputs  between  the  minds,  and  the  nature  of  the  mind’s  response  dependent  on  these  factors.  The 
detail  he  provides  is  beyond  what  is  needed  to  gain  perspective  on  the  mind’s  working  as  it 
affects  intelligence  analysis.  For  our  purpose,  Stanovich  establishes  the  following  attributes  of 
the  mind: 

•  Humans  by  nature  are  cognitive  misers.  We  are  inclined  to  lessen  our  cognitive 
load  and  the  need  for  information.  This  is  Type  1  processing  which  is  the  default 
mode.  Unless  intervention  occurs  from  other  parts  of  the  mind,  Type  1  processes 
will  dominate  the  mind’s  response. 

•  An  important  feature  of  the  algorithmic  mind  is  to  override  the  processing  being 
done  by  the  autonomous  mind  before  the  results  of  Type  1  processes  constitute  the 
output  of  the  mind. 

•  There  are  few  differences  between  the  autonomous  minds  of  individuals. 
Differences  in  the  algorithmic  minds  of  individuals  “reflects  reasoning  abilities 


22  Stanovich  p.  22. 

23  Stanovich  p.  22. 

24  Stanovich  p.  39. 


Distribution  A:  Approved  for  public  release;  distribution  unlimited. 
88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


operating  across  of  a  variety  of  domains”  Individual  differences  in  the  reflective 
mind  depend  on  thinking  dispositions  and  the  inputs  to  the  functioning  of  the 
reflective  mind;  these  inputs  are  called  the  mindware. 

•  The  inputs  to  each  of  the  functional  partitions  of  the  mind  are: 

>  Autonomous  mind  -  information  or  knowledge  acquired  from  evolution  and 
procedures  which  have  been  acquired  through  practice. 

>  Algorithmic  mind  -  strategies  and  processes  for  processing  of  information, 
e.g.,  procedural  knowledge. 

>  Reflective  mind  -  beliefs,  declarative  knowledge,  and  goals. 

We’ll  now  focus  the  rest  of  the  discussion  on  the  nature  of  the  reflective  mind,  discussing 
thinking  dispositions  and  mindware. 

3.2.3.  Thinking  Dispositions 

What  are  thinking  dispositions?  This  is  a  term  widely  used  in  cognitive  psychology  to  cover  a 
wide  range  of  cognitive  behavior.  Examples  of  thinking  dispositions  and  their  associated 
cognitive  propensities  are  “actively  open  minded  thinking,  the  need  for  cognition  (the  tendency 
to  think  a  lot),  consideration  of  future  consequences,  need  for  closure,  superstitious  thinking,  and 

dogmatism . the  tendency  to  collect  information  before  making  up  one’s  mind,  the  tendency  to 

seek  various  points  of  view  before  coming  to  a  conclusion,  the  disposition  to  think  extensively 
about  a  problem  before  responding,  tendency  to  calibrate  the  degree  of  strength  of  one's  opinion 
to  the  degree  of  evidence  available,  the  tendency  to  think  about  future  consequences  before 
taking  action,  the  tendency  to  explicitly  weigh  pluses  and  minuses  of  situations  before  making  a 
decision,  and  the  tendency  to  seek  nuance  and  avoid  absolutism.”  (Stanovich  2009:21-22). 

Another  set  of  thinking  dispositions  associated  with  critical  thinking  are: 

•  Persisting  -  Stick  to  it. 

•  Thinking  and  communicating  with  clarity  and  precision  -  Be  clear. 

•  Managing  impulsivity  -  Take  your  time. 

•  Gathering  data  through  all  senses  -  Use  your  natural  pathways. 

•  Listening  with  understanding  and  empathy  -  Understand  others. 

•  Creating,  imagining,  innovating  -  Try  a  different  way. 

•  Thinking  flexibly  -  Look  at  it  another  way. 

•  Responding  with  wonderment  and  awe  -  Have  fun  figuring  it  out. 

•  Thinking  about  your  thinking  (metacognition)  -  Know  your  knowing. 

•  Taking  responsible  risks  -  Venture  out. 

•  Striving  for  accuracy  and  precision  -  Check  it  again. 

•  Binding  humor  -  Laugh  a  little. 

•  Questioning  and  problem  posing  -  How  do  you  know. 

•  Thinking  interdependently  -  Work  together. 


25  Stanovich  p.  13. 

26  Stanovich  pp.  21-22. 
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•  Applying  past  knowledge  to  new  situations  -  Use  what  you  learn. 

97 

•  Remaining  open  to  continuous  learning  -  Learn  from  experiences" 

The  preceding  thinking  dispositions  are  the  manifestation  of  a  very  active  mind.  A  very  active 
mind  accepts  new  information,  subjects  that  new  information  to  scrutiny,  determines  the  new 
information’s  consistency  with  previously  acquired  information,  and  evaluates  the  consistency  of 
the  conclusions  drawn  from  previous  infonnation  with  those  drawn  from  the  new  information. 

3.2.4.  Mindware 

As  we  previously  stated,  the  three  inputs  to  the  reflective  mind  are:  beliefs,  declarative 
knowledge,  and  goals.  Stanovich  labels  this  collection  of  inputs  to  the  reflective  mind  as 
mindware,  based  on  a  concept  developed  by  Perkins.  By  doing  so,  he  has  provided  a  definition 
of  the  content  of  the  mind.  This  definition  pertains  to  the  inputs  to  the  mind’s  cognition.  It  does 
not,  however,  relate  to  other  attributes  of  an  individual  that  govern  the  intensity,  frequency,  and 
continuity  of  applying  cognition. 

Within  the  intelligence  literature,  the  role  of  the  content  of  the  mind  in  intelligence  analysis  has 
been  a  frequent  topic.  The  literature  recognizes  the  relationship  between  analyst  judgments  and 
what  is  called  mindset.  Similarly,  the  cognitive  psychology  literature  recognizes  terms  like 
mental  model  and  framing,  and  those  terms  have  been  used  to  convey  the  concept  of  the  content 
of  the  mind.  What  has  been  lacking,  however,  has  been  an  explicit  identification  of  the  inputs  to 
the  mind  that  constitute  the  content  of  the  mind.  Thus,  Stanovich’s  definition  represents  a 
significant  finding:  by  identifying  the  inputs,  you  can  focus  efforts  to  improve  the  content  of  the 
mind.  Towards  this  end,  we  elaborate  on  each  of  the  inputs  to  the  reflective  mind  -  beliefs, 
declarative  knowledge,  and  goals  -  in  the  following. 

3.2.5.  Beliefs 

One  very  important  consequence  of  thinking  dispositions  is  the  establishment  of  beliefs.  The 
Merriam- Webster  online  dictionary  defines  belief  as  “conviction  of  the  truth  of  some  statement 
or  the  reality  of  some  being  or  phenomenon  especially  when  based  on  examination  of  evidence.” 
The  implication  of  this  definition  is  that  beliefs  are  the  product  of  reasoning  and  thought. 
However,  increasingly  the  view  is  held  that  two  types  of  beliefs  exist.  Abelson  put  forth  this 
view: 

But  it  is  important  to  distinguish  between  testable  belief,  belief  about  objects  within  the 
immediate  experience  of  the  person  that  allow  appropriate  action  and  feedback,  and  distal 

29 

belief,  belief  about  objects  only  remotely  experienced  or  not  sensibly  verifiable." 

In  the  more  recent  literature,  beliefs  that  are  not  founded  on  an  examination  of  evidence  are 
called  memes.  Memes  are  given  many  of  the  attributes  of  genes,  i.e.,  their  basic  objective  is  to 
replicate  and  survive.  As  a  result,  these  types  of  beliefs  are  passed  from  one  individual  to 
another  without  proof  and  without  conscious  awareness.  Some  have  tenned  memes  as  the  virus 


27  Costa,  A.L.  &  Kallick,  B.  Describing  16  Habits  of  Mind,  The  Institute  for  Habits  of  Mind  [undated]. 

28  Perkins,  D.N.  Outsmarting  IQ:  The  Emerging  Science  ofLearnable  Intelligence,  Free  Press,  New  York,  NY, 

1995. 

29  Abelson,  R.P.  "Beliefs  are  Like  Possessions,"  Journal  for  the  Theory’  of  Social  Behaviour,  Vol.  16,  No.  3,  1986,  p. 
229. 
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of  the  mind.  Stanovich  devotes  an  entire  chapter  to  memes  and  identifies  them  as  contaminated 

*31 

mindware  since  they  can  cause  thinking  that  is  not  rational. 

Memes  (and  beliefs  referred  to  by  the  common  use  of  the  term  bias,  discussed  earlier)  result  in 
irrationality.  Cognitive  biases  also  result  in  irrationality.  However,  the  basis  for  cognitive  bias  is 
different  than  the  basis  for  these  irrational  beliefs.  Consequently,  the  actions  you  take  to  mitigate 
the  negative  effects  of  each  are  somewhat  different.  As  discussed  earlier,  humans  are  by  nature 
cognitive  misers  and  inclined  to  lessen  our  cognitive  load  and  need  for  information. 
Consequently,  cognitive  biases  will  naturally  evidence  themselves  if  one  does  not  stop  and  think. 
Rather  than  immediately  and  automatically  responding  to  a  consideration  or  question,  however, 
you  can  learn  to  recognize  and  override  these  tendencies  resulting  from  Type  1  processing  in  the 
autonomous  mind. 

Dealing  with  irrational  beliefs  can  be  much  more  difficult  than  dealing  with  cognitive  biases. 

The  role  of  beliefs  in  intelligence  analysis  is  substantial.  We  all  have  beliefs  that  can  affect  our 
analysis.  You  must  employ  appropriate  thinking  dispositions  and  actively  engage  the  reflective 
mind  to  deal  with  irrational  beliefs.  First,  you  must  try  to  develop  a  keen  awareness  of  the  beliefs 
you  have.  However,  awareness  of  your  beliefs  is  not  sufficient.  You  must  consider  the  potential 
effects  of  those  beliefs  on  your  analysis.  Then  you  should  make  your  beliefs  and  the  possible 
effects  of  those  beliefs  explicitly  visible  to  yourself  and  others.  This  will  enable  you  and  others 
to  better  understand  your  thinking  and  your  output.  The  dialogue  generated  with  others  will  help 
you  challenge  your  own  thinking. 

In  the  intelligence  literature,  the  word  “assumptions”  is  used  instead  of  beliefs.  This  difference 
in  terminology  has  major  ramifications  since  the  cognitive  and  remedial  activity  associated  with 
each  term  is  different.  For  example,  “assumptions”  can  be  propositions  that  are  considered  as 
true  for  the  purpose  of  guiding  investigation.  If  this  is  the  case,  then  such  assumptions  can  be 
listed  and  changed  relatively  easily  as  more  information  is  acquired  and  explicit  cognition  is 
applied.  However,  if  the  “assumption”  is  based  on  an  analyst’s  underlying  beliefs  (e.g.,  memes, 
common  biases),  the  remedial  action  (as  described  above)  is  much  more  difficult  because  the 
belief  is  held  without  being  subject  to  explicit  cognition.  Because  “assumptions”  (using  the 
common  language  in  the  intelligence  literature)  can  include  beliefs  and  are  much  more  than 
propositions  that  will  guide  investigation  and  subsequently  be  changed,  deliberate  efforts  are 
needed  to  subject  your  beliefs  to  scrutiny.  Even  so,  listing  all  of  the  beliefs  used  in  the  analysis 
of  information  after  the  fact  is  extremely  difficult.  This  difficulty  of  knowing  what  beliefs  are 
used  in  the  analysis  can  be  mitigated  by  identifying  the  beliefs  used  in  assigning  meaning  to 
information  at  the  time  the  meaning  is  assigned. 

3.2.6.  Knowledge 

The  second  input  to  the  reflective  mind  is  declarative  knowledge.  Pirolli  defined  declarative 
knowledge  as  follows: 

Declarative  knowledge  is  the  kind  of  knowledge  that  a  person  can  attend  to,  reflect  upon, 
and  usually  articulate  in  some  way  (e.g.,  by  declaring  it  verbally  or  by  gesture). 


30  Brodie,  R.  Virus  of  the  Mind:  The  New  Science  of  the  Meme,  Integral  Press,  Seattle,  WA,  1996. 

31  Stanovich  pp.  152-171. 
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Declarative  knowledge  includes  the  kinds  of  factual  knowledge  that  users  can  verbalize, 
such  as  "The  ’open’  item  on  the  ’file’  menu  will  open  a  file.32 

In  other  words,  declarative  knowledge  is  what  we  know;  the  more  we  know,  the  greater  the 
potential  for  better  thinking. 

Procedural  knowledge  is  not  part  of  mindware,  but  is  an  important  component  of  the  mind.  It  is 
defined  as  follows: 

Procedural  knowledge  is  defined  as  the  strategies  and  sequences  of  operations  used  in 
problem-solving  (know-how).33 

Sternberg  says  that  procedural  knowledge  is  tacit  knowledge  and  “takes  the  form  of  rules  of 
thumb  for  what  to  do  under  what  circumstances.”34  This  function  supports  the  algorithmic  mind. 
As  such,  procedural  knowledge  is  an  important  contribution  to  cognition.  With  effort,  one  can 
improve  their  procedural  knowledge.  But,  we  will  not  deal  with  this  since  the  focus  of  our 
discussion  is  the  functioning  of  the  reflective  mind. 

The  general  role  of  knowledge  is  well  recognized  in  the  intelligence  literature.  This  is 
manifested  by  recommendations  for  intensive  training  and  education  of  analysts.  However, 
much  less  emphasis  is  placed  on  developing  the  deep  knowledge  associated  with  expertise.  This 
is  also  evidenced  in  practice  by  the  diminished  emphasis  on  long-term  analysis. 

3.2.7.  Goals 

The  last  input  to  the  mindware  is  goals.  As  mentioned  in  the  “Introduction  to  the  FAC”  section, 
the  meaning  of  information  is  based  on  the  purpose  for  which  it  is  being  analyzed.  The  purpose 
of  looking  at  any  information  establishes  the  goal  as  it  is  represented  in  the  mindware. 

While  the  intelligence  literature  highlights  the  importance  of  knowing  the  requirement  for 
intelligence  analysis,  why  you  are  doing  the  analysis  is  infrequently  covered  in  this  literature.  As 
a  result,  little  emphasis  has  been  placed  on  the  importance  of  knowing  why  you  are  doing  the 
analysis  as  a  means  of  improving  your  understanding  of  the  purpose  of  analysis.  We  will  cover 
this  extensively  in  the  “Developing  Intelligence  Issue  Statements”  section  of  the  FAC. 

3.2.8.  So  What? 

The  preceding  discussion  provides  a  model  of  how  the  mind  works  and  the  information  used  by 
the  mind  to  produce  its  responses.  Though  many  cognitive  psychologists  would  likely  find  this 
model  an  over-simplification,  Stanovich’s  views  are  accurately  reflected,  albeit  in  summary 
form.  Moreover,  this  model  does  provide  an  understanding  with  practical  ramifications.  The 
model  serves  as  a  basis  for  actions  on  a  part  of  an  individual  that  will  result  in  greater  rationality 
and  more  nuanced  judgments.  These  are  the  major  practical  insights  from  this  model: 

•  You  have  to  work  at  being  rational.  People  have  the  potential  for  being  both  rational 
and  irrational,  but  the  default  mode  of  providing  responses  with  minimum  cognitive 

32  Pirolli,  P.  Information  Foraging  Theory’:  Adaptive  Interaction  with  Information,  Oxford  University  Press,  Oxford, 
UK,  2007,  p.  9. 

33  Chao,  C.  &  Salvendy,  G.  "Impact  of  Cognitive  Abilities  of  Experts  on  the  Effectiveness  of  Elicited  Knowledge," 
Behaviour  &  Information  Technology,  Vol.  14,  No.  3,  1995,  p.  174. 

34  Sternberg,  R.J.  "Epilogue:  What  do  we  Know  about  Tacit  Knowledge?  Making  the  Tacit  Become  Explicit,"  in 
Tacit  Knowledge  in  Professional  Practice:  Researcher  and  Practitioner  Perspectives,  eds.  R.J.  Sternberg  &  J.A. 
Horvath,  Lawrence  Erlbaum  Publishers,  Mahwah,  NJ,  1999,  p.  231. 
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effort  is  likely  to  result  in  irrational  thought.  This  is  not  the  consequence  of  a  defect 
or  a  limitation  of  the  mind  as  these  tenns  are  normally  used.  The  default  mode  of 
irrationality  is  the  natural  consequence  of  the  autonomous  mind,  which  is  the  result 
of  evolutionary  factors. 

Few  differences  exist  in  the  autonomous  mind  between  individuals  except  for  the 
infonnation  learned  from  extensive  practice  such  as  playing  golf,  driving  a  car,  or 
visual  inspections. 

The  capability  of  the  algorithmic  mind  is  determined  by  its  ability  to  execute  tasks 
analogous  to  a  computer.  This  ability  is  largely  innate  but  can  be  enhanced  by 
learning  various  processing  techniques  and  other  procedural  knowledge. 

The  reflective  mind  is  the  primary  source  for  rational  thought  which  is  largely 
determined  by  the  mindware  it  uses  to  perfonn  its  functions. 

Mindware  is  probably  the  most  important  factor  in  producing  rational  thought. 
While  mindware  does  not  have  inherent  limitations,  gaps  and  contamination  in  the 
mindware  are  a  source  of  irrationality.  Fortunately,  mindware  is  the  most 
susceptible  part  of  the  mind  to  efforts  to  improve  its  quality  and  to  address  gaps  and 
contamination. 

Thinking  dispositions  associated  with  rational  thought  are  known.  For  some,  the 
use  of  an  appropriate  thinking  disposition  will  naturally  occur.  But,  almost 
everyone  will  benefit  by  learning  the  positive  thinking  dispositions  and  applying 
them  during  the  formulation  of  thought. 

Beliefs  have  a  very  powerful  influence  on  rational  thought.  It  is  extremely  difficult 
for  an  individual  to  inventory  their  beliefs.  However,  with  effort  you  can  identify 
the  beliefs  that  exhibit  a  major  role  in  your  thinking.  You  need  to  scrutinize  these 
beliefs  to  detennine  if  they  are  valid  or  should  be  replaced  by  new  beliefs.  The 
recipients  of  your  thinking,  whether  yourself  or  others,  need  to  be  infonned  of  the 
beliefs  that  were  instrumental  in  forming  your  judgments. 

Understanding  purpose  is  instrumental  to  your  thinking.  Your  thinking  will  reflect 
your  understanding  of  your  purpose.  An  incomplete  or  inaccurate  understanding  of 
purpose  will  likely  result  in  thinking  that  is  not  relevant  to  the  issue  to  be  addressed. 
An  important  characteristic  of  purpose  is  that  it  is  not  a  static  condition  and  will 
vary  as  the  problem  being  thought  about  changes.  Literally,  every  time  one  starts 
thinking  on  a  topic  or  condition,  the  purpose  for  thinking  about  the  topic  or 
condition  will  have  to  be  established. 

It  is  highly  likely  that  substantial  differences  exist  in  the  mindware  and  procedural 
knowledge  between  individuals.  These  differences  account  for  the  often  observed 
fact  that  the  same  information  can  have  different  meaning  to  different  individuals. 

Mindware  is  not  static  but  continually  changes.  When  significant  changes  occur, 
the  possibility  exists  that  the  meaning  assigned  to  information  based  on  prior 
mindware  will  require  re-analysis. 

The  existence  of  a  default  mode  (i.e.,  the  autonomous  mind)  does  not  preordain  a 
person  to  irrational  thoughts.  Other  partitions  of  the  mind  exist,  the  algorithmic  and 
reflective,  that  can  override  the  action  of  the  autonomous  mind  and  produce  a 
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rational  response.  With  awareness  and  effort,  one  can  produce  rational  output  from 
the  mind. 


Applying  the  preceding  information  does  not  guarantee  rationality  in  thought.  However,  by 
applying  this  information,  the  probability  of  rational  thought  will  be  significantly  enhanced. 

Consequently,  this  information  provides  the  guidelines  for  improved  analytical  results. 

3.3  Knowing  Yourself 

The  prior  portion  of  this  section  dealt  with  irrationality,  whether  based  on  cognitive  biases  or 
irrational  beliefs.  The  purpose  of  this  portion  of  the  section  is  to  describe  some  of  the  most 
frequently  observed  irrational  behaviors.  The  ability  to  recognize  behaviors  as  irrational  is  the 
first  step  in  efforts  to  preclude  such  behavior. 

3.3.1.  Cognitive  Biases 

As  stated  earlier,  a  very  large  body  of  research  has  been  conducted  to  show  the  behavior  of 
people  that  deviates  from  rationality.  While  some  controversy  exists  regarding  the  nature  of  this 
research,  most  people  under  a  variety  of  circumstances  exhibit  these  cognitive  biases.  Appendix 
A  is  a  listing  of  the  cognitive  biases  identified  from  this  research  and  is  provided  for  your 
awareness. 

The  many  ways  people  can  deviate  from  rationality  is  evident  by  the  extent  of  this  list.  Not  all 
people  show  these  biases  under  all  circumstances.  However,  under  most  circumstances,  the 
preponderance  of  people  behaves  as  described.  The  magnitude  of  these  results  validates  the 
observation  that  "If  you  are  like  most  people,  then  like  most  people,  you  don’t  know  you're  like 

35 

most  people." 

3.3.2.  Performance  Factors 

In  addition  to  the  above  cognitive  biases,  other  aspects  of  behavior  have  been  observed  that 
definitely  affect  one’s  thinking.  Nearly  all  of  the  following  behaviors  stem  from  faulty 
mindware,  particularly  from  beliefs. 

3.3.3.  Self-Appraisal 

The  first  thing  you  need  to  recognize  is  that  you  may  not  be  as  competent  as  you  think  you  are. 
You  have  all  likely  heard  of  the  Lake  Wobegon  effect  where  all  the  students  are  above  average. 
One  thinks  of  this  as  a  comedian’s  line,  but  in  effect  this  reflects  a  real  condition  of  personal  self 
evaluation.  Numerous  studies  have  been  done  on  self  appraisal  on  a  variety  of  performance 
attributes,  and  the  results  are  consistent  in  all  of  these  studies.  A  high  majority  of  the  people  rate 
themselves  as  being  above  average. 

One  of  the  most  revealing  studies  was  performed  by  Kruger  and  Dunnings  in  which  participants 
were  asked  to  assess  their  ability  in  areas  related  to  knowledge  and  wisdom,  humor,  logical 
reasoning,  and  writing.  In  addition,  the  participants  took  a  test  in  each  of  these  areas  that  was 
designed  to  measure  their  ability.  The  test  results  were  scored  by  people  deemed  experts  in  each 
of  these  fields.  The  results  were  consistent  with  other  similar  research.  On  average,  the 


35  Gilbert  as  cited  by  Stanovich  p.  201. 

36  Kruger,  J.  &  Dunning,  D.  "Unskilled  and  Unaware  of  it:  How  Difficulties  in  Recognizing  One's  Own 
Incompetence  Lead  to  Inflated  Self-Assessments,"  Journal  of  Personality  and  Social  Psychology’,  Vol.  77,  No.  6, 
1999,  pp.  1121-1134. 
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participants  rated  themselves  as  above  average  in  ability;  but  the  test  results  demonstrated 
abilities  inconsistent  with  the  self  appraisal.  Additional  insight  was  gained  by  examining  the 
results  on  an  individual  basis.  Two  noteworthy  results  were  that  some  people,  generally  less 
than  25%  of  the  participants,  were  able  to  accurately  assess  their  abilities.  In  contrast,  the  gap 
between  the  self  assessment  and  the  measured  assessment  was  greatest  for  the  people  with  the 
lowest  ability. 

The  results  were  all  the  more  surprising  given  the  nature  of  the  participants.  They  were  all 
Cornell  University  undergraduates.  Since  the  general  acceptance  rate  at  Cornell  is  about  30%, 
the  participants  hardly  represent  the  distribution  of  ability  in  the  general  population.  A  general 
correlation  between  the  SAT  scores  of  Cornell  students  and  Intelligence  Quotient  (IQ)  indicates 
that  the  students  are  in  the  upper  20  percentile  of  the  population.  Thus,  perhaps  only  10%  of  the 
general  population  may  be  able  to  accurately  assess  their  own  abilities. 

The  scope  of  inquiry  was  also  expanded  by  giving  the  participants  examples  of  competent  results 
of  grammar  use.  The  least  competent  participants  were  unable  to  recognize  competence. 

Indeed,  the  lowest  rated  participants  tended  to  raise  their  appraisal  of  their  own  competence 
when  confronted  with  competent  results.  Thus  the  researchers  concluded  that  the  least 
competent  are  unable  to  recognize  competence  when  they  see  it.  These  results  seem  to  confirm 
the  observation: 

In  short,  one’s  knowledge  of  the  limitations  of  one's  knowledge  is  itself  necessarily 
limited  by  one's  knowledge:  The  less  one  knows,  the  less  one  can  be  aware  of  how  much 
one  does  not  know. 

The  last  finding  has  application  in  the  intelligence  analysis  process.  A  natural  part  of  this 
process  is  the  review  and  critique  of  an  analyst’s  product.  Instinctively,  we  may  not  readily 
accept  critical  comments  on  our  efforts.  The  work  of  Kruger  and  Dunning  suggests  that  we 
should  not  immediately  reject  substantive  comments.  In  doing  so,  we  might  be  rejecting  a 
learning  experience.  Thoughtful  consideration  in  assessing  the  comments  is  in  order  since  the 
person  making  those  comments  may  know  more  than  we. 

3.3.4.  Peer  Assessment 

As  described  above,  despite  the  opportunity  to  have  a  wealth  of  knowledge  about  themselves,  a 
large  number  of  people  are  unable  to  accurately  appraise  their  own  capabilities.  In  stark  contrast 
with  these  results  are  those  that  show  that  an  individual’s  capabilities  can  be  accurately,  in  many 
case  more  accurately,  assessed  by  others.  Even  total  strangers  provided  with  a  limited  amount 
of  information  can  perform  an  accurate  assessment.  The  behaviors  better  predicted  by  others 
cover  a  wide  spectrum.  They  include  almost  all  of  those  behaviors  which  individuals 
inaccurately  predict  relative  to  themselves.  These  behaviors  include  prediction  of  promotion  into 
a  leadership  position,  scoring  on  a  test  of  surgical  skills,  the  duration  of  a  romantic  relationship, 
donation  to  charitable  causes,  and  time  to  accomplish  a  task.  In  the  work  environment,  the  offset 
between  self  assessment  and  the  assessment  of  others  includes  those  of  both  coworkers  and 


37  Nickerson,  R.S.  "How  we  Know  -  and  Sometimes  Misjudge  -  what  Others  Know:  Imputing  One's  Own 
Knowledge  to  Others,"  Psychological  Bulletin,  Vol.  125,  No.  6,  1999,  p.  749. 

38  Dunning,  D.,  Heath,  C.  &  Suls,  J.M.  "Flawed  Self-Assessment:  Implications  for  Health,  Education,  and  the 
Workplace,"  Psychological  Science  in  the  Public  Interest,  Vol.  5,  No.  3,  2004,  p.  71. 
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supervisors.  The  possibility  that  they  know  you  better  than  you  know  yourself  should  be  a 
consideration  in  your  interaction  with  others. 

2.3.5.  Physical  Aspects 

You’re  no  doubt  familiar  with  the  phrase  “sound  mind,  sound  body.”  Recent  research  indicates 
that  Buddha  had  an  accurate  understanding  of  the  relationship: 

To  keep  the  body  in  good  health  is  a  duty...  otherwise  we  shall  not  be  able  to  keep  our 
mind  strong  and  clear. 

Contrary  to  conventional  wisdom  that  thinking  is  not  hard  work,  an  increasing  amount  of 
research  shows  that  your  brain  can  become  tired.39  This  research  also  shows  that  a  tired  brain 
does  not  function  as  well  as  a  rested  brain.  The  state  of  the  brain,  tired  or  rested,  affects  the 
decision  making  function  of  the  brain;  tests  show  poorer  decisions  are  made  with  a  tired  brain 
than  with  a  rested  brain. 

The  prescription  for  resting  your  brain  is  little  different  than  that  for  resting  a  muscle:  give  it  a 
rest.  However,  for  the  brain  this  is  easier  said  than  done  since  not  thinking  is  a  difficult  state  to 
achieve.  Interestingly,  the  environment  in  which  you  are  placed  does  have  a  significant  effect  on 
the  recovery  of  the  brain.  Research  has  shown  that  interacting  with  nature  significantly  improves 
the  cognitive  function.40  In  contrast,  strolling  in  a  city  has  less  of  restorative  effect  on  the  brain. 
The  difference  has  been  attributed  to  the  fact  that  an  urban  environment  demands  your  attention 
as  a  matter  of  routine.  For  example,  strolling  in  a  city  requires  that  your  brain  be  constantly 
engaged  to  avoid  cars  and  people  and  to  arrive  at  the  desired  destination. 

Not  surprisingly,  sleep  is  another  means  of  achieving  a  rested  brain.  In  addition,  sleep  also 
affects  our  memories  in  terms  of  consolidation  and  learning.  The  most  recent  research  shows 
that  while  sleep  in  general  is  good,  the  beneficial  impact  is  much  higher  for  specific  types  of 
sleep.  This  research  provides  evidence  supporting  the  anecdotal  stories  of  how  major  scientific 
discoveries  occurred  after  sleep.  According  to  the  research,  deep  Rapid  Eye  Movement  (REM) 
sleep  will  significantly  improve  perfonnance  on  creative  problem  solving  tasks.  Test  subjects 
that  had  non-REM  sleep  or  rest  did  not  show  a  corresponding  improvement  in  performance.41 
On  the  negative  side,  a  lack  of  sleep  will  produce  false  memories.42 

Nutrition  can  also  restore  a  tired  brain.  People  were  given  tasks  that  required  lots  of  thinking  and 
reasoning.  Consistent  with  other  research,  the  people  with  a  tired  brain  did  not  make  effective 
decisions.  However,  with  an  increase  in  blood-sugar  levels  the  brain  functions  were  restored  and 
effective  decisions  were  made.43 


39  Amir,  O.  "Tough  Choices:  How  Making  Decisions  Tires  Your  Brain",  Scientific  American:  Mind  Matters, 
[Online],  2008.  Available:  http://www. scientificamerican. com/ article. cfm?id=tough-choices-how-making 

40  Lehrer,  J.  "The  Cognitive  Benefits  of  Nature",  [Online],  2008.  Available: 
http://scienceblogs.com/cortex/2008/ll/the_cognitive_benefits_of_natu.php 

41  Cai,  D.,  Mednick,  S.,  Harrison,  E.,  Kanady,  J.  &  Mednick,  S.  "REM,  Not  Incubation,  Improves  Creativity  by 
Priming  Associative  Networks,"  Proceedings  of  the  National  Academy  of  Sciences,  Vol.  106,  No.  25,  2009,  pp. 
10130-10134. 

42  Lehrer,  J.  "Don't  Trust  an  Insomniac",  [Online],  2008,  Available: 
http://scienceblogs.eom/cortex/2008/l  l/dont_trust_an_insomniac.php 

43  Anonymous.  “Sugaring  the  Decision”  Economist,  Vol.  386,  No.  8573,  pp.  104-106. 
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3.3.6.  External  Influences 


A  topic  of  considerable  research  is  what  is  called  the  “self-fulfilling  prophecy  effect.”  The  self- 
fulfilling  prophecy  effect  is  when  a  person’s  belief  about  another’s  capabilities  will  elicit 
behavior  consistent  with  that  belief.  This  effect  has  been  observed  over  a  wide  variety  of 
subjects  and  topics.  A  notable  example  was  a  study  on  teachers  and  their  interactions  with 
students.  A  randomly  selected  group  of  students  were  identified  as  having  above  average  IQ.  At 
the  end  of  a  year,  those  so  identified  showed  more  progress  than  their  fellow  students  not  so 
identified.44 

So  regardless  of  your  opinion  of  yourself,  your  coworkers’  or  supervisors’  opinions  of  your 
capabilities  could  enhance  or  degrade  your  perfonnance.  Of  course,  this  is  a  reciprocal  effect. 
Your  opinion  of  your  coworkers  or  supervisors  could  also  be  affecting  their  performance.  This 
could  easily  become  a  mutually  reinforcing  self-fulfilling  prophecy  effect  where  both  parties  are 
diminishing  the  performance  of  the  other  party,  a  very  negative  consequence  in  the  workplace. 
The  research  findings  on  this  topic  support  Cooper’s  observation: 

One  of  the  best  established  findings  in  social  psychology  is  that  the  expectations  one 
person  holds  for  another  can  affect  the  other  person's  behavior.45 

A  similar  effect  can  be  observed  with  people  who  are  members  of  a  stereotyped  group.  If  they 
are  reminded  that  they  are  part  of  such  a  group,  their  perfonnance  will  degrade  consistent  with 
the  stereotype.  In  contrast,  people  not  so  reminded  will  perform  contrary  to  the  stereotype.46 

Others  can  affect  your  behavior  in  even  more  subtle  ways.  Being  exposed  to  a  random  list  of 
words  that  contain  a  preponderance  of  words  relating  to  a  specific  behavior  will  cause  one  to 
adapt  and  behave  consistently  with  those  words.  For  example,  when  non-elderly  people  were 
exposed  to  words  associated  with  the  attributes  of  being  elderly,  they  physically  mimicked  the 
behavior  of  the  elderly.47  In  a  similar  experiment,  people  exposed  to  words  associated  with 
either  courteous  or  aggressive  behavior  subsequently  acted  in  a  manner  consistent  with  the  words 
they  had  read. 

Groups  can  also  intensify  the  strength  of  your  beliefs.  If  you  spend  some  time  in  a  discussion 
with  a  group  who  largely  believes  the  same  as  you  do,  after  these  discussions  your  beliefs  will 
not  only  be  stronger  but  more  extreme  in  their  interpretation.48 

3.3.7.  Internal  Influences 

As  part  of  the  state  of  your  mind,  you  have  an  extensive  set  of  beliefs  that  govern  how  you 
interpret  information.  If  you  are  convinced  that  a  relationship  exists  between  cost  and  quality 
your  judgment  of  all  products  will  be  so  influenced.  This  has  been  demonstrated  in  many 
different  products  and  circumstances.  For  example,  when  people  are  given  the  same  wine  with 
one  sample  labeled  as  less  expensive  than  another  sample,  they  consistently  judge  the  more 


44  Olson,  J.M.,  Roese,  N.J.  &  Zanna,  M.P.  "Expectancies,"  in  Social  Psychology’:  Handbook  of  Basic  Principles, 
Eds.  E.T.  Higgins  &  A.W.  Kmglanski,  Guilford  Press,  New  York,  NY,  1996,  p.  222. 

45  Cooper,  H.M.  Synthesizing  Research:  A  Guide  for  Literature  Reviews,  3rd  Edn,  Sage  Publications,  Thousand 
Oaks,  CA,  1998,  p.  10. 

46Arielyp.  168. 

47  Ariely  p.  171. 

48  Sunstein,  C.R.  Why  Societies  Need  Dissent,  Harvard  University  Press,  Cambridge,  MA,  2003,  p.  111. 
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expensively  labeled  wine  as  superior49  A  similar  effect  occurred  when  wine  glasses  were 
evaluated  even  though  taste  tests  detected  no  difference  as  a  function  of  the  glasses  used  for 
drinking.50  Another  example  showed  that  people  consistently  preferred  Coke  over  Pepsi  when 
they  could  see  the  labels.  When  the  labels  were  removed,  Pepsi  was  the  preferred  drink.51  The 
same  effect  has  even  been  found  in  medicines.  People  reported  different  reactions  from  the  same 
pill  depending  on  the  price  associated  with  the  pill.52 

The  most  famous  of  the  self-influence  effects  is  the  placebo  effect.  Innumerable  tests  have  been 
conducted  about  patient  reactions  to  treatment  when  they  thought  they  were  getting  the 
treatment.  The  effect  has  not  only  been  demonstrated  for  medicines  but  many  other  types  of 
treatment  including  surgery.  How  the  placebo  effect  works  is  not  well  understood,  but  the 
effect  demonstrates  how  our  mind  assigns  meaning  to  information  that  then  controls  our  body. 

Of  all  the  personal  beliefs  that  affect  one’s  perfonnance  one  of  the  most  surprising  is  the  effect 
from  one’s  belief  in  the  nature  of  human  intelligence.  Some  believe  that  intelligence  is  a  fixed 
trait  determined  by  inherited  genes.  The  opposite  belief  is  that  intelligence  can  be  improved  by 
the  appropriate  environment.  These  two  beliefs  are  widely  held  and  debated,  although  some 
such  as  Stanovich  believe  that  the  debate  is  over  even  if  the  public  is  unaware  of  it.54 

Dweck55  measured  the  performance  of  two  groups  with  people  of  equivalent  levels  of 
intelligence  based  on  IQ  tests.  The  groups  had  different  views  on  the  nature  of  intelligence.  One 
group  believed  that  a  person’s  intelligence  is  fixed  and  detennined  by  one’s  inherited  genes.  The 
other  group  believed  in  a  malleable  concept  of  intelligence,  i.e.,  that  it  can  be  increased  by 
appropriate  effort.  Those  who  believed  that  intelligence  is  fixed  were  discouraged  by  failure  in 
cognitive  tests,  lost  confidence  in  themselves,  and  avoided  challenging  situations.  Even  their 
performance  was  affected  by  failure.  Often  if  they  retook  a  test  that  they  successfully  completed 
before  experiencing  failure,  they  would  do  less  well  than  originally.  The  opposite  behavior  was 
noted  for  those  that  believed  in  malleable  intelligence.  They  believed  that  effort  is  important  and 
were  willing  to  put  forth  an  increasing  amount  of  effort.  The  groups  even  differed  in  the  way 
they  responded  to  praise.  When  the  group  that  believed  intelligence  was  fixed  was  complimented 
on  being  smart,  their  subsequent  performance  was  reduced.  In  contrast,  praise  for  efforts  by  the 
group  believing  in  malleable  intelligence  resulted  in  increase  subsequent  perfonnance. 

3.3.8.  So  What  -  Again 

As  mentioned  previously,  the  behaviors  identified  in  this  portion  of  the  section  are  not  inevitable. 
One  is  not  predestined  to  exhibit  any  of  these  behaviors.  In  fact,  an  interesting  aspect  of  the 
research  that  has  identified  these  behaviors  is  that  a  percentage  of  the  participants,  usually  less 
that  25%,  behave  in  a  rational  manner.  Research  is  underway  to  understand  why  these  people 


49  Goldstein,  R.,  Almenberg,  J.,  Dreber,  A.,  Emerson,  J.W.,  Herschkowitsch,  A.  &  Katz,  J.  Do  More  Expensive 
Wines  Taste  Better?  Evidence  from  a  Large  Sample  of  Blind  Tastings,  American  Association  of  Wine  Economists 
(AAWE)  Working  Paper  No.  16,  Walla  Walla,  WA,  2008. 

50Arielyp.  165. 

51  Ariely  p.  166. 

52  Ariely  p.  181. 

53  Ariely  p.  174. 

54  Stanovich,  K.E.  What  Intelligence  Tests  Miss:  The  Psychology  of  Rational  Thought,  Yale  University  Press,  New 
Haven,  CT,  2009,  p.  20. 

55  Dweck,  C.S.  "Beliefs  that  Make  Smart  People  Dumb,"  in  Why  Smart  People  can  be  so  Stupid,  ed.  R.J.  Sternberg, 
Yale  University  Press,  New  Haven,  CT,  2002,  pp.  24-41. 
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behave  differently.  The  results  of  this  research  are  generally  inconclusive,  but  the  behavior 
doesn’t  seem  to  correlate  with  the  IQs  of  the  participants.  Being  smarter  does  not  necessarily 
protect  one  from  irrational  behavior. 

But  even  without  the  instincts  of  those  who  seem  to  be  able  avoid  irrational  behavior,  you  can 
emulate  their  performance  with  knowledge  of  what  irrationality  is  and  of  its  sources.  A  variety 
of  techniques  are  available  that  will  help  ameliorate  tendencies  towards  irrationality.  Applying 
these  techniques  requires  both  awareness  and  the  expenditure  of  considerable  effort.  One  such 
technique,  Writing  As  Thinking,  is  discussed  in  the  next  portion  of  this  section.  Throughout  the 
FAC,  we  will  continue  to  provide  information  to  increase  your  knowledge  and  awareness  to  help 
you  achieve  more  rational  analysis. 

Forewarned,  forearmed:  to  be  prepared  is  half  the  victory 

—Miguel  de  Cervantes 

3.4  Writing  as  Thinking:  Using  Written  Expression  to  Improve  Your  Analysis 

Don  7  think  and  then  write  it  down.  Think  on  paper. 

—  Harry  Kemelman 

The  best  reason  for  putting  anything  down  on  paper  is  that  one  may  then  change  it. 

—  Bernard  De  Voto 


3.4.1.  Introduction 

In  this  portion  of  the  FAC,  we  discuss  the  concept  of  writing  as  thinking,  or  epistemic  writing. 
Epistemic  writing  is  writing  that  not  only  results  from  thought,  but  writing  that  become  “an 
integral  part  of  thought.”56  Writing  can  be  used  not  only  to  communicate  with  others,  but  also  to 
help  you  generate  ideas,  to  engage  in  self-reflection  as  you  develop  your  thoughts  about  a  topic, 
and  to  connect  and  develop  those  ideas  into  coherent  themes  and  assessments.  Epistemic  writing 
is  more  than  open-ended  idea  generation  or  simple  note -taking.  Being  self-reflective  aids  you  in 
considering  new  ideas  not  considered  before.  It  promotes  further  examination  of  your  thoughts, 
resulting  in  an  increased  richness  and  clarity  of  analytic  understanding.  In  effect,  writing  as 
thinking  is  communicating  with  yourself.  As  we  discussed  in  the  “Thinking  about  Thinking” 
and  “Knowing  Yourself’  portions  of  this  section  of  the  FAC,  it  is  necessary  for  you  to 
deliberately  reflect  upon  and  develop  the  content  of  your  mind.  Writing  can  be  an  important  aid 
to  these  cognitive  efforts. 

In  studying  writers’  comments  about  writing  in  publications  outside  of  intelligence  analysis,  the 
experience  of  learning  or  creating  something  new  while  authoring  a  work  seems  to  be  almost 
universal  among  accomplished  writers.  The  act  of  writing  is  more  than  just  creating  a  readable 
text;  it  becomes  a  means  for  writers  to  enlarge  their  understanding  of  what  they  are  trying  to 
communicate. 

In  addition,  writing  is  a  means  of  making  your  thoughts  concrete  so  that  you  can  analyze  them 
further.  When  your  thinking  is  all  in  your  head,  it  is  easy  to  ignore  flaws  in  your  mindware.  That 
is,  you  lack  what  is  known  as  metacognitive  distance57  from  your  ideas  when  they  are 

56  Bereiter,  C.  &  Scardamalia,  M.  The  Psychology  of  Written  Composition,  L.  Erlbaum  Associates,  Hillsdale,  NJ, 
1987,  p.  88. 

57  Dunn,  P.A.  Talking,  Sketching,  Moving:  Multiple  Literacies  in  the  Teaching  of  Writing,  Boynton/Cook, 
Portsmouth,  NH,  200 1 . 
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unexpressed.  Metacognitive  distance  is  the  process  of  stepping  outside  your  own  ideas  in  order 
to  analyze  and  reshape  your  thinking.  Once  your  ideas  are  captured  in  writing,  you  can  think 
about  the  thinking  that  the  written  word  represents  and  take  a  closer  look  at  your  assumptions, 
beliefs,  and  knowledge.  Writing  can  become  an  important  strategy  to  mitigate  many  of  the 
cognitive  pitfalls  you  may  encounter  during  intelligence  analysis. 

While  the  role  of  writing  as  thinking  is  widely  recognized  in  the  general  literature  about 
composition,  the  use  of  writing  to  aid  cognition  is  not  acknowledged  in  the  intelligence  literature. 
The  research  we  conducted  before  developing  the  FAC  determined  that  the  intelligence  analysis 
literature  views  writing  as  primarily  a  means  of  creating  a  final  product  for  the  customer.  The 
intelligence  analysis  literature  does  not  appear  to  recognize  that  writing  can  be  integrated  with 
thinking  and  that  writing  done  with  this  approach  has  the  potential  to  improve  the  cognition 
associated  with  intelligence  analysis. 

Authors  not  only  relegate  writing  to  the  later  stages  of  intelligence  analysis  but  also  consider  it 
primarily  the  production  of  grammatically  correct,  easy  to  read  and  understand  deliverables.  For 
example,  this  is  the  description  of  writing  provided  in  a  table  labeled  “Cognitive  Abilities 
Required  of  Intelligence  Analysts:” 

Written  Expression:  The  ability  to  use  words  and  sentences  in  writing  so  others  will 
understand.  Involves  knowledge  of  the  meanings  and  distinctions  among  words, 

58 

knowledge  of  grammar,  and  the  ability  to  organize  sentences  and  paragraphs. 

Other  articles  in  intelligence  publications  give  only  a  passing  mention  to  writing  when  discussing 
the  skills  that  analysts  need.  Moore,  Krizan,  and  Moore  mention  “literacy”59  in  a  chart  with  no 
elaboration  in  the  text.  Moore  and  Krizan60  mention  that  analysts  need  basic  writing  skills  but 
that  they  often  lack  them. 

In  this  portion  of  the  FAC  we  present  some  background  infonnation  about  the  role  of  written 
expression  in  thinking,  present  the  use  of  written  expression  as  a  form  of  discovery,  and  discuss 
how  written  expression  is  integrated  into  the  FAC. 

3.4.2.  Knowledge  Telling  vs.  Knowledge  Transformation 

Researchers  who  study  written  composition  make  a  distinction  between  two  types  of  written 
expression:  knowledge  telling  and  knowledge  transformation .  Knowledge  telling  is  the 
production  of  text  which  is  on  topic,  but  which  relies  on  the  writer’s  readily  available 
knowledge.  Knowledge  telling  “preserves  the  straight-ahead  form  of  oral  language  production 
and  requires  no  significantly  greater  amount  of  planning  or  goal-setting  than  does  ordinary 
conversation.”61  Knowledge  telling  writing  strategies  are  common  in  elementary  or  middle 
school  students,  but  also  persist  in  high  school  and  college  writing  assignments.  Many  adult 
writers  in  business  or  professional  settings  also  use  a  knowledge  telling  approach  in  carrying  out 
their  responsibilities. 


58  Krizan.  L.  Intelligence  Essentials  for  Everyone,  Joint  Military  Intelligence  College,  Washington,  D.C.,  1999,  p. 
56. 

59  Moore,  D.T.,  Krizan,  L.  &  Moore,  E.J.  "Evaluating  Intelligence:  A  Competency-Based  Model,"  International 
Journal  of  Intelligence  and  Counterintelligence,  Vol.  18,  No.  2,  2005,  p.  206. 

60  Moore,  D.T.  &  Krizan,  L.  "Intelligence  Analysis:  Does  NSA  have  what  it  Takes?"  Cryptologic  Quarterly,  Vol. 
20,  No.  1-2,  2001,  p.  17. 

61  Bereiter,  C.  &  Scardamalia,  M.  The  Psychology’  of  Written  Composition,  L.  Erlbaum  Associates,  Hillsdale,  NJ, 
1987,  p.10. 
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In  contrast,  knowledge  transformation  is  thinking  “in  which  the  thoughts  come  into  existence 
through  the  composing  process  itself,  beginning  as  inchoate  entities  (“driblets”)  and  gradually, 
by  dint  of  much  rethinking  and  restating,  taking  the  form  of  fully  developed  thoughts.”  "  Written 
expression  that  results  in  knowledge  transfonnation  has  been  observed  to  include  problem¬ 
solving  and  planning  activities.  The  writer  is  involved  in  “deliberately  fonnulating  and  pursuing 
personally  meaningful  goals  in  writing,  for  recognizing  and  overcoming  problems,  and  for 
assessing  and  revising  choices  made  at  a  variety  of  levels.”63 

The  distinction  between  knowledge  telling  and  knowledge  transformation  in  intelligence  analysis 
is  an  important  one.  As  we  mentioned  in  the  Introduction  and  Background  Section  of  the  FAC, 
unless  analysis  results  in  something  that  was  not  explicitly  contained  in  the  information  under 
analysis,  analysis  has  not  been  done;  the  result  is  reporting.  One  of  our  goals  in  recommending 
the  use  of  writing  to  aid  thinking  is  to  encourage  a  knowledge  transforming  approach  to 
intelligence  analysis.  The  next  part  discusses  the  specific  points  in  the  FAC  where  knowledge 
transformation  may  take  place. 

3.4.3.  Epistemic  Writing  and  the  FAC 

We  believe  that  there  are  six  key  cognitive  challenges  that  intelligence  analysts  face  during 
intelligence  analysis.  They  are: 

•  Discerning  the  intelligence  issue 

•  Translating  the  intelligence  issue  into  a  plan  for  analysis 

•  Transfonning  the  intelligence  issue  into  a  search  strategy 

•  Assigning  meaning  to  or  deriving  meaning  from  the  information  obtained 

•  Marshalling  of  the  evidence  to  arrive  at  an  answer  to  the  issue  being  addressed 

•  Communicating  the  results  of  the  analysis  effort  to  the  customer 

When  you  are  faced  with  these  cognitive  challenges,  how  do  you  accomplish  each  of  them?  For 
example,  consider  the  challenge  of  discerning  an  intelligence  issue,  whether  coming  from  a 
customer  or  initiated  by  you.  We  reviewed  a  recent  document  that  discussed  STA  techniques.  It 
suggested  that  describing  the  customer’s  problem  could  be  accomplished  in  “5-10  minutes.”64 
The  authors  of  this  document  recommend  thinking  about  the  customer’s  problem  using  a  few 
questions,  but  didn’t  investigate  how  an  analyst  might  engage  more  deeply  with  the  customer’s 
problem.  The  model  for  understanding  a  customer’s  intelligence  issue  in  this  approach  is  based 
on  knowledge  telling:  the  analyst  uses  the  information  he  or  she  has  on  hand  to  conceptualize  the 
customer’s  intelligence  issue. 

The  FAC,  however,  relies  on  writing  as  one  of  the  key  cognitive  supports  for  addressing  these 
analytic  challenges.  Using  the  example  concerning  the  customer’s  problem  given  above,  the 
guidelines  in  the  FAC  encourage  you  to  work  towards  understanding  the  customer’s  problem  in  a 
variety  of  ways,  including  writing  down  your  thinking  and  questions  about  the  customer’s 
intelligence  issue.  Initially,  writing  down  your  thoughts  will  serve  to  develop  your 
understanding  and  questions  about  the  intelligence  issue.  This  thinking,  expressed  in  writing, 
will  be  used  as  you  interact  with  the  customer  to  better  understand  his  or  her  needs  and 


62  Bereiterp.  10. 

63  Bereiterp.  10. 

64  Anonymous.  A  Tradecraft  Primer:  Structured  Analytic  Techniques  for  Improving  Intelligence  Analysis  2009,  p.  5. 
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perspective.  Ultimately,  the  written  representation  of  the  issue  becomes  the  means  by  which  you 
and  the  customer  reach  a  mutual  understanding  of  the  issue.  The  FAC  provides  you  with 
suggestions  and  guides  that  not  only  help  you  assess  your  current  knowledge  but  also  extend 
your  knowledge.  These  activities  encourage  you  to  engage  in  the  issue;  writing  becomes  a  path 
to  knowledge  transformation. 

The  majority  of  the  writing  you  do  to  address  the  above  challenges  will  be  only  for  yourself. 
Some  of  the  ideas  and  text  that  you  create  will  be  used  again  when  you  write  the  product  that  you 
will  deliver  to  the  customer.  At  times  you  may  share  some  of  this  writing  with  the  customer  or 
your  colleagues  for  discussion  and  feedback  but  they  are  created  by  you,  for  you,  to  help  you 
think  better  about  the  analysis  work  you  are  doing. 

Finally,  the  FAC  does  not  prescribe  any  one  method  of  using  writing  as  thinking.  What  works 
for  one  person  creates  writer’s  block,  and  therefore  thinker’s  block,  for  another.  For  example,  no 
one  environment  has  proven  to  be  the  best  for  writing.  Surveys  of  professional  authors  have 
found  a  wide  range  of  preferences  for  noise,  time  of  day,  writing  instruments,  setting,  clothing, 
paper,  and  technology.  Some  authors’  preferences  are  rooted  in  superstition  and  long  habit.  The 
same  can  be  said  for  note-taking,  outlining,  research,  revision,  and  editing.  We  encourage  you  to 
discover  what  works  best  for  you.  The  key  is  to  engage  in  self-reflection  and  develop  and 
connect  ideas  into  coherent  themes  using  whatever  methods  work  for  you. 

In  the  following  sections  we  will  discuss  some  concepts  related  to  writing  as  thinking  that 
explain  the  topic  further.  These  include  the  ideas  of  idea  generation,  text  generation,  and  the  role 
of  revision  in  epistemic  writing. 

3.4.4.  Idea  Generation  vs.  Text  Generation 

If  you  accept  the  challenge  of  using  writing  as  a  means  of  thinking,  you  may  need  to  view  the 
steps  you  use  to  create  a  written  piece  differently.  Instead  of  knowing  what  you  want  to  say 
before  you  say  it,  for  example,  you  may  need  to  adopt  the  viewpoint  of  many  skilled  writers: 
they  consider  writing  as  a  “voyage  into  the  unknown.”65 

Once  you’ve  undertaken  the  voyage,  writing  helps  you  both  capture  your  ideas  and  then 
ultimately  communicate  them  to  your  audience.  One  strategy  for  using  writing  to  improve  your 
thinking  is  to  separate  idea  generation  from  text  generation.  This  is  especially  helpful  in  the 
early  stages  of  writing.  Collins  and  Gentner  discuss  the  role  of  idea  generation  and  text 
generation  in  their  framework  for  a  cognitive  theory  of  writing: 

The  importance  of  the  distinction  between  producing  ideas  and  producing  text  cannot  be 
overemphasized. .  .one  of  the  most  damaging  habits  for  a  novice  writer  to  have  is  that  of 
confusing  idea  manipulation  with  text  manipulation  so  that  text  structure  constraints  enter 
into  the  process  of  writing  at  an  early  stage,  before  the  ideas  are  ready.  When  this 
happens,  not  only  does  the  writer  waste  a  great  deal  of  time  and  effort  polishing  prose 
that  will  eventually  be  discarded,  but,  even  worse,  the  effort  to  perfect  text  may  cause  the 
writer  to  lose  track  of  the  desired  content.  Part  of  the  reason  that  novice  writers  have 
difficulty  accepting  the  distinction  between  idea  production  and  text  production  is  that  it 
is  very  unintuitive  to  realize  the  degree  to  which  one's  own  ideas  can  be  opaque, 
inaccessible,  self-contradictory,  and  in  other  ways  in  need  of  considerable  strategic 


65  Murray,  D.M.  Write  to  Learn,  8th  Edn,  Thomson  Wadsworth,  Boston,  MA,  2005. 
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management  in  order  to  emerge  as  the  clear,  well-reasoned  positions  we  believe  we 
have,  [emphasis  added]66 

Intelligence  analysis  requires  both  idea  generation  and  text  generation.  However,  in  order  to  be 
effective,  it  also  requires  particular  kinds  of  idea  generation  and  text  generation.  “Writing  as 
Thinking”  focuses  on  idea  generation.  In  the  “Analysis  Representation”  section  we  will  discuss 
bringing  all  the  various  written  representations  that  you  develop  throughout  the  FAC  into  an 
integrated  form.  Text  generation  for  the  purpose  of  communicating  to  your  customers  will  be 
covered  in  the  “Customer  Product”  section  of  the  FAC.  The  following  discussion  presents  more 
details  about  idea  generation  and  other  aspects  of  writing  as  thinking. 

3.4.5.  Two  Dimensions  of  Idea  Generation 

If  you  write  while  you  are  generating  ideas,  your  writing  plays  two  important  roles  in  your 
thinking.  Writing  down  an  idea  as  it  occurs  captures  it  as  a  tangible  artifact.  Once  the  artifact  is 
created,  you  have  a  medium  for  exploring  an  idea  further,  adding  new  knowledge  to  an  idea, 
creating  other  new  ideas,  or  even  rejecting  the  idea  once  you  have  pursued  it  further  in  writing. 
Written  expression  becomes  the  “externalization  of  cognition”  [emphasis  added]  as  described 
in  the  following  passage: 

One  way  to  overcome  the  difficulties  of  performing  such  complex  knowledge 
manipulation  in  the  head  is  to  capture  ideas  on  paper  (or  some  other  external  medium 
such  as  a  computer  screen)  in  the  fonn  of  external  representations  that  stand  for  mental 
structures.  So  long  as  ideas,  plans,  and  drafts  are  locked  inside  a  writer's  head,  then 
modifying  and  developing  them  will  overload  the  writer's  short-term  memory... Writing 
creates  external  representations  and  the  external  representations  condition  the  writing 
process.  Cognition  is  not  simply  expressed  or  amplified  through  the  use  of  external 
representations,  but  rather  the  nature  of  thought  is  detennined  by  the  mind's  dialectical 
interaction  with  the  world  as  constructed  by  human  beings  (Kuutti,  1991;  Wood,  1992). 
Notes,  sketches,  outlines,  tables,  topic  lists,  concept  maps,  and  argument  structures  are 
both  representations  of  mental  content  and  things  in  themselves,  new  stimuli  dissociated 
from  the  moment  of  their  production  and  available  for  reinterpretation.67 

Creating  a  written  record  of  ideas  not  only  captures  them,  but  makes  them  available  as  material 
for  further  exploration  during  a  cognitive  effort.  The  written  representation  frees  the  brain  from 
trying  to  remember  ideas  as  they  occur  and  provides  new  material  for  more  thinking.  The  ideas 
you  generate  during  intelligence  analysis  can  be  captured  as  short  notes,  charts,  sketches, 
diagrams,  tables  -  not  just  paragraphs  consisting  of  complete  sentences.  The  tools  or  supplies 
you  use  to  write  down  and  store  the  ideas  are  also  not  crucial  to  better  cognition.  The  most 
critical  writing  task  at  each  phase  of  the  FAC  is  that  you  capture  the  ideas  you  are  generating  in 
some  form  so  that  they  are  available  for  further  reference  and  thought. 


66  Collins,  A.  &  Centner,  D.  "A  Framework  for  a  Cognitive  Theory  of  Writing,"  in  Cognitive  Processes  in  Writing, 
eds.  L.W.  Gregg  &  E.R.  Steinberg,  L.  Erlbaum  Associates,  Hillsdale,  NJ,  1980,  p.  53. 

67  Sharpies,  M.  (1996).  “An  Account  of  Writing  as  Creative  Design”  in  The  Science  of  Writing:  Theories,  Methods, 
Individual  Differences,  and  Applications,  eds.  C.  M.  Levy  &  S.  E.  Ransdell,  L.  Erlbaum,  Mahwah,  NJ,  pp.  127-148. 
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3.4.6.  The  Role  of  Revision  in  Epistemic  Writing 

Once  an  idea  becomes  a  tangible  artifact,  you  can  think  about  the  thinking  that  it  represents. 
Writing  gives  you  metacognitive  distance  from  an  idea  and  a  method  for  re-thinking  what  you 
have  written:  revision.  Our  research  has  provided  more  insight  into  two  styles  of  revision: 

Internal  Revision.  Under  this  term,  I  include  everything  writers  do  to  discover  and 
develop  what  they  have  to  say,  beginning  with  the  reading  of  a  completed  first  draft. 

They  read  to  discover  where  their  content,  fonn,  language,  and  voice  have  led  them. 

They  use  language,  structure,  and  information  to  find  out  what  they  have  to  say  or  hope 
to  say.  The  audience  is  one  person:  the  writer. 

External  Revision.  This  is  what  writers  do  to  communicate  what  they  have  found  they 
have  written  to  another  audience.  It  is  editing  and  proofreading  and  much  more.  Writers 
now  pay  attention  to  the  conventions  of  form  and  language,  mechanics,  and  style. 

(Murray  1978)68 

You  will  notice  throughout  the  FAC  that,  whenever  a  written  artifact  is  recommended,  we  will 
encourage  you  to  focus  on  internal  revision  when  you  use  the  written  artifact  to  develop  your 
own  thinking.  You  will  concentrate  on  external  revision  when  you  are  communicating  the 
understanding  that  you’ve  developed.  We  supply  guides  to  help  you  get  your  original  ideas 
down  on  paper,  we  recommend  obtaining  feedback  and  more  infonnation,  and  we  advise  you  to 
identify  your  target  audience  and  create  documents  that  meet  that  audience’s  needs. 

3.4.7.  Summary:  Writing  as  Thinking  in  Intelligence  Analysis 

The  intelligence  literature  and  tradition  is  focused  on  writing  as  the  final  stage  of  intelligence 
analysis.  However,  thinking  occurs  throughout  analysis.  This  section  of  the  FAC  puts  forth  the 
concept  of  writing  for  yourself  as  a  means  of  capturing  your  thinking  about  a  topic,  manipulating 
those  thoughts  to  create  new  thoughts,  and  validating  your  thinking  once  you  have  expressed  it  in 
writing.  Writing  as  thinking  is  essential  to  help  you  develop  ideas  into  coherent  themes  and 
assessments  and  to  encourage  you  to  engage  in  self-reflection  about  the  development  of  your 
thoughts.  Writing  is  necessary  to  help  you  deliberately  reflect  upon  and  develop  the  contents  of 
your  mind  as  we  discussed  in  the  “Thinking  about  Thinking”  and  “Knowing  Yourself’  sections 
of  the  FAC.  All  of  this  activity  takes  place  before  you  communicate  your  understanding  of  an 
issue  to  other  analysts  or  to  your  customer. 

Besides  helping  your  own  analysis  efforts,  writing  as  thinking  can  benefit  others  that  work  with 
you.  When  you  capture  your  thoughts  during  all  stages  of  intelligence  analysis,  you  will 
ultimately  be  better  able  to  communicate  your  thinking  to  your  colleagues  and  customers.  They 
in  turn  will  be  better  able  to  understand  how  you  arrived  at  the  judgments  you  develop. 

The  FAC  is  intended  to  help  build  your  metacognitive  awareness  -  your  ability  to  separate  what 
you  think  from  how  you  think.  Using  writing  as  thinking  first  captures  what  you  think  and  then 
lets  you  analyze  your  written  thoughts  to  evaluate  how  you  are  thinking.  No  other  cognitive  aid 
offers  this  powerful  support  to  intelligence  analysis  and  the  cognitive  challenges  it  represents. 


68  Murray,  D.  M.  (1978).  “Internal  Revision:  a  Process  of  Discovery”  in  Research  on  Composing:  Points  of 
Departure,  eds.  C.  R.  Cooper  &  L.  Odell,  National  Council  of  Teachers  of  English,  Urbana,  IL,  pp.  85-103. 
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Writing  is  easy.  You  just  sit  down  at  a  typewriter  and  open  a  vein. 

—Red  Smith 

3.5  Cognitive  Biases69  -  A  Supplement 

The  following  sections  elaborate  the  biases  identified  by  researchers.  We  provide  these  as  a 
reference  and  to  demonstrate  the  wide  variety  of  biases  that  have  been  identified  in  publications 
outside  of  intelligence  analysis. 

3.5.1.  Decision-Making  and  Behavioral  Biases 

•  Bandwagon  effect  —  the  tendency  to  do  (or  believe)  things  because  many  other 
people  do  (or  believe)  the  same. 

>  Base  rate  fallacy  —  ignoring  available  statistical  data  in  favor  of  particulars. 

•  Bias  blind  spot  —  the  tendency  not  to  compensate  for  one's  own  cognitive  biases. 

•  Choice-supportive  bias  —  the  tendency  to  remember  one's  choices  as  better  than 
they  actually  were. 

•  Confirmation  bias  —  the  tendency  to  search  for  or  interpret  information  in  a  way 
that  confirms  one's  preconceptions. 

•  Congruence  bias  —  the  tendency  to  test  hypotheses  exclusively  through  direct 
testing,  in  contrast  to  tests  of  possible  alternative  hypotheses. 

•  Conservatism  bias  —  the  tendency  to  ignore  the  consequence  of  new  evidence. 

•  Contrast  effect  —  the  enhancement  or  diminishing  of  a  weight  or  other 
measurement  when  compared  with  a  recently  observed  contrasting  object. 

•  Deformation  professionnelle  —  the  tendency  to  look  at  things  according  to  the 
conventions  of  one's  own  profession,  forgetting  any  broader  point  of  view. 

•  Distinction  bias  —  the  tendency  to  view  two  options  as  more  dissimilar  when 
evaluating  them  simultaneously  than  when  evaluating  them  separately. 

•  Endowment  effect  —  "the  fact  that  people  often  demand  much  more  to  give  up  an 
object  than  they  would  be  willing  to  pay  to  acquire  it". 

•  Experimenter's  or  Expectation  bias  —  the  tendency  for  experimenters  to  believe, 
certify,  and  publish  data  that  agree  with  their  expectations  for  the  outcome  of  an 
experiment,  and  to  disbelieve,  discard,  or  downgrade  the  corresponding  weightings 
for  data  that  appear  to  conflict  with  those  expectations. 

•  Extraordinarily  bias  —  the  tendency  to  value  an  object  more  than  others  in  the  same 
category  as  a  result  of  an  extraordinarity  of  that  object  that  does  not,  in  itself, 
change  the  value. 

•  Extreme  aversion  —  the  tendency  to  avoid  extremes,  being  more  likely  to  choose 
an  option  if  it  is  the  intennediate  choice. 


69  List  of  Cognitive  Biases.  Wikipedia,  5  May  2009. 
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•  Focusing  effect  —  prediction  bias  occurring  when  people  place  too  much 
importance  on  one  aspect  of  an  event;  causes  error  in  accurately  predicting  the 
utility  of  a  future  outcome. 

•  Framing  —  by  using  a  too  narrow  approach  or  description  of  the  situation  or  issue. 
Also  framing  effect  —  drawing  different  conclusions  based  on  how  data  are 
presented. 

•  Hyperbolic  discounting  —  the  tendency  for  people  to  have  a  stronger  preference  for 
more  immediate  payoffs  relative  to  later  payoffs,  where  the  tendency  increases  the 
closer  to  the  present  both  payoffs  are. 

•  Illusion  of  control  —  the  tendency  for  human  beings  to  believe  they  can  control  or 
at  least  influence  outcomes  that  they  clearly  cannot. 

•  Impact  bias  —  the  tendency  for  people  to  overestimate  the  length  or  the  intensity  of 
the  impact  of  future  feeling  states. 

•  Information  bias  —  the  tendency  to  seek  information  even  when  it  cannot  affect 
action. 

•  Irrational  escalation  —  the  tendency  to  make  irrational  decisions  based  upon 
rational  decisions  in  the  past  or  to  justify  actions  already  taken. 

•  Loss  aversion  —  "the  disutility  of  giving  up  an  object  is  greater  than  the  utility 
associated  with  acquiring  it". 

•  Mere  exposure  effect  —  the  tendency  for  people  to  express  undue  liking  for  things 
merely  because  they  are  familiar  with  them. 

•  Moral  credential  effect  —  the  tendency  of  a  track  record  of  non-prejudice  to 
increase  subsequent  prejudice. 

•  Need  for  closure  —  the  need  to  reach  a  verdict  in  important  matters;  to  have  an 
answer  and  to  escape  the  feeling  of  doubt  and  uncertainty.  The  personal  context 
(time  or  social  pressure)  might  increase  this  bias. 

•  Neglect  of  probability  —  the  tendency  to  completely  disregard  probability  when 
making  a  decision  under  uncertainty. 

•  Not  Invented  Here  —  the  tendency  to  ignore  that  a  product  or  solution  already 
exists,  because  its  source  is  seen  as  an  "enemy"  or  as  "inferior". 

•  Omission  bias  —  the  tendency  to  judge  harmful  actions  as  worse,  or  less  moral, 
than  equally  harmful  omissions  (inactions). 

•  Outcome  bias  —  the  tendency  to  judge  a  decision  by  its  eventual  outcome  instead 
of  based  on  the  quality  of  the  decision  at  the  time  it  was  made. 

•  Planning  fallacy  —  the  tendency  to  underestimate  task-completion  times. 

•  Post-purchase  rationalization  —  the  tendency  to  persuade  oneself  through  rational 
argument  that  a  purchase  was  a  good  value. 

•  Pseudocertainty  effect  —  the  tendency  to  make  risk-averse  choices  if  the  expected 
outcome  is  positive,  but  make  risk-seeking  choices  to  avoid  negative  outcomes. 

•  Reactance  —  the  urge  to  do  the  opposite  of  what  someone  wants  you  to  do  out  of  a 
need  to  resist  a  perceived  attempt  to  constrain  your  freedom  of  choice. 

38 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 

88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


•  Selective  perception  —  the  tendency  for  expectations  to  affect  perception. 

•  Status  quo  bias  —  the  tendency  for  people  to  like  things  to  stay  relatively  the  same 
(see  also  loss  aversion,  endowment  effect,  and  system  justification). 

•  Von  Restorff  effect  —  the  tendency  for  an  item  that  "stands  out  like  a  sore  thumb" 
to  be  more  likely  to  be  remembered  than  other  items. 

•  Wishful  thinking  —  the  fonnation  of  beliefs  and  the  making  of  decisions  according 
to  what  is  pleasing  to  imagine  instead  of  by  appeal  to  evidence  or  rationality. 

•  Zero-risk  bias  —  preference  for  reducing  a  small  risk  to  zero  over  a  greater 
reduction  in  a  larger  risk. 

3.5.2.  Biases  in  Probability  and  Belief 

•  Ambiguity  effect  —  the  avoidance  of  options  for  which  missing  infonnation  makes 
the  probability  seem  "unknown". 

•  Anchoring  —  the  tendency  to  rely  too  heavily,  or  "anchor,"  on  a  past  reference  or 
on  one  trait  or  piece  of  information  when  making  decisions. 

•  Attentional  bias  —  neglect  of  relevant  data  when  making  judgments  of  a  correlation 
or  association. 

•  Authority  bias  —  the  tendency  to  value  an  ambiguous  stimulus  (e.g.,  an  art 
perfonnance)  according  to  the  opinion  of  someone  who  is  seen  as  an  authority  on 
the  topic. 

•  Availability  heuristic  —  estimating  what  is  more  likely  by  what  is  more  available  in 
memory,  which  is  biased  toward  vivid,  unusual,  or  emotionally  charged  examples. 

•  Availability  cascade  —  a  self-reinforcing  process  in  which  a  collective  belief  gains 
more  and  more  plausibility  through  its  increasing  repetition  in  public  discourse  (or 
"repeat  something  long  enough  and  it  will  become  true"). 

•  Clustering  illusion  —  the  tendency  to  see  patterns  where  actually  none  exist. 

•  Capability  bias  —  the  tendency  to  believe  that  the  closer  average  perfonnance  is  to 
a  target,  the  tighter  the  distribution  of  the  data  set. 

•  Conjunction  fallacy  —  the  tendency  to  assume  that  specific  conditions  are  more 
probable  than  general  ones. 

•  Gambler's  fallacy  —  the  tendency  to  think  that  future  probabilities  are  altered  by 
past  events,  when  in  reality  they  are  unchanged.  Results  from  an  erroneous 
conceptualization  of  the  normal  distribution. 

•  Hawthorne  effect  —  the  tendency  of  people  to  perform  or  perceive  differently  when 
they  know  that  they  are  being  observed. 

•  Hindsight  bias  —  sometimes  called  the  "I-knew-it-all-along"  effect,  the  inclination 
to  see  past  events  as  being  predictable. 

•  Illusory  correlation  —  beliefs  that  inaccurately  suppose  a  relationship  between  a 
certain  type  of  action  and  an  effect. 
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•  Ludic  fallacy  —  the  analysis  of  chance  related  problems  according  to  the  belief  that 
the  unstructured  randomness  found  in  life  resembles  the  structured  randomness 
found  in  games,  ignoring  the  non-gaussian  distribution  of  many  real-world  results. 

•  Neglect  of  prior  base  rates  effect  —  the  tendency  to  neglect  known  odds  when 
reevaluating  odds  in  light  of  weak  evidence. 

•  Observer-expectancy  effect  —  when  a  researcher  expects  a  given  result  and 
therefore  unconsciously  manipulates  an  experiment  or  misinterprets  data  in  order  to 
find  it  (see  also  subject-expectancy  effect). 

•  Optimism  bias  —  the  systematic  tendency  to  be  over-optimistic  about  the  outcome 
of  planned  actions. 

•  Ostrich  effect  —  ignoring  an  obvious  (negative)  situation. 

•  Overconfidence  effect  —  excessive  confidence  in  one's  own  answers  to  questions. 

•  Positive  outcome  bias  —  a  tendency  in  prediction  to  overestimate  the  probability  of 
good  things  happening  to  them. 

•  Pareidolia,  vague  and  random  stimulus  (often  an  image  or  sound)  are  perceived  as 
significant,  e.g.,  seeing  images  of  animals  or  faces  in  clouds,  the  man  in  the  moon, 
and  hearing  hidden  messages  on  records  played  in  reverse. 

•  Primacy  effect  —  the  tendency  to  weigh  initial  events  more  than  subsequent  events. 

•  Recency  effect  —  the  tendency  to  weigh  recent  events  more  than  earlier  events 

•  Disregard  of  regression  toward  the  mean  —  the  tendency  to  expect  extreme 
perfonnance  to  continue. 

•  Reminiscence  bump  —  the  effect  that  people  tend  to  recall  more  personal  events 
from  adolescence  and  early  adulthood  than  from  other  lifetime  periods. 

•  Rosy  retrospection  —  the  tendency  to  rate  past  events  more  positively  than  they  had 
actually  rated  them  when  the  event  occurred. 

•  Selection  bias  —  a  distortion  of  evidence  or  data  that  arises  from  the  way  that  the 
data  are  collected. 

•  Stereotyping  —  expecting  a  member  of  a  group  to  have  certain  characteristics 
without  having  actual  infonnation  about  that  individual. 

•  Subadditivity  effect  —  the  tendency  to  judge  probability  of  the  whole  to  be  less 
than  the  probabilities  of  the  parts. 

•  Subjective  validation  —  perception  that  something  is  true  if  a  subject's  belief 
demands  it  to  be  true.  Also  assigns  perceived  connections  between  coincidences. 

•  Telescoping  effect  —  the  effect  that  recent  events  appear  to  have  occurred  more 
remotely  and  remote  events  appear  to  have  occurred  more  recently. 

•  Texas  sharpshooter  fallacy  —  the  fallacy  of  selecting  or  adjusting  a  hypothesis  after 
the  data  is  collected,  making  it  impossible  to  test  the  hypothesis  fairly.  Refers  to  the 
concept  of  firing  shots  at  a  bam  door,  drawing  a  circle  around  the  best  group,  and 
declaring  that  to  be  the  target. 
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4.0  DEVELOPING  INTELLIGENCE  ISSUE  STATEMENTS 

...you  cannot  gather  information  meaningfully  unless  you  have  understood  the  problem  but. ..you 
cannot  understand  the  problem  without  information  about  it... 

—  Rittel70 


4.1  Introduction 

Would  you  be  concerned  if  a  doctor  asked  you  the  reason  for  your  current  appointment  and  then 
began  treatment  without  asking  you  any  additional  questions?  Suppose  you  commissioned  an 
architect  to  create  plans  for  a  new  house.  Would  you  be  nervous  if  the  architect  instructed  your 
builder  to  begin  construction  before  you  had  a  chance  to  talk  to  the  architect  about  what  you 
wanted?  In  real  life,  neither  professional  would  proceed  without  interaction  with  the  patient  or 
client  early  in  the  process.  The  goal  of  the  interaction  is  to  develop  a  shared  representation  of  the 
issue  at  hand  before  the  professional  applies  his  or  her  professional  knowledge.  Arriving  at  a 
shared  representation  may  be  a  messy  and  circular  process,  but  expert  practitioners  would  rarely, 
if  ever,  skip  this  step. 

Intelligence  analysis  also  has  a  “getting  started”  period.  This  getting  started  period  can  occur  in 
several  ways:  (1)  when  you  are  assigned  to  a  new  Analytic  Area  of  Responsibility  (AOR)  by 
your  organization,  i.e.,  an  AOR  issue ;  (2)  when  a  question  or  issue  is  initiated  by  you  for 
analysis,  i.e.,  a  self-initiated  issue ;  or  (3)  when  someone  else  initiates  a  question  for  you  to 
address,  i.e.,  a  external  issue.  In  the  guidance  and  examples  that  follow,  we  will  address  these 
three  types  of  issues. 

We  recommend  that  you  engage  in  thorough  questioning  and  examination  of  the  issue  for  all 
types  of  issues.  For  example,  if  others  within  your  or  another  intelligence  organization  initiate, 
then  creating  a  shared  representation  of  the  issue  with  them  is  important.  They  are  the  customer, 
and  you  can  use  the  processes  described  in  the  following  to  attain  a  shared  representation  of  the 
issue.  You  can  also  use  developing  your  issue  statement  as  a  means  to  identify  prospective 
customers  that  are  not  in  intelligence  organizations.  Sound  professional  practice  suggests  that 
the  intelligence  analyst  have  a  thorough  understanding  of  any  issue,  as  well  as  a  shared 
representation  of  the  issue  with  any  actual  or  prospective  customers,  just  as  a  doctor  or  architect 
would  with  their  clients. 

The  intelligence  analysts  we  consulted  to  write  the  FAC  offered  this  rule  of  thumb  about 
intelligence  questions  and  issues:  the  stated  issue  is  not  always  the  issue.  Your  actual  or 
prospective  customer  may  be  very  perceptive,  but  his  or  her  view  of  the  issue  is  unlikely  to  be 
comprehensive  in  terms  of  understanding  the  issue.  It’s  the  goal  of  this  FAC  section  to  provide 
you  with  a  process  and  guides  so  that  you  can  effectively  determine  what  the  real  issues  are 
behind  the  intelligence  issues  that  you  are  asked  to  address  or  that  you  ask  yourself. 

In  the  FAC,  we  refer  to  this  thorough,  shared  representation  as  an  intelligence  issue  statement. 
We  refer  to  the  process  used  to  explore  an  intelligence  area  or  question  as  developing  the 
intelligence  issue  statement. 


70  Rittel,  H.W.J.,  Grant,  D.P.  &  Protzen,  J.  (1984).  “Second-generation  Design  Methods'’  in  Developments  in  Design 
Methodology’,  ed.  N.  Cross,  Wiley,  New  York,  NY,  pp.  321. 
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Who  is  a  customer? 


In  the  FAC,  the  intended  audience  for  any  of  the  three  types  of  intelligence  issues  is  referred  to 
as  the  customer.  For  example,  with  an  AOR  issue,  one  intended  audience  is  your  management; 
you  want  to  create  a  shared  representation  with  them  about  the  scope  of  your  analytic 
responsibilities.  In  addition,  those  who  will  eventually  use  the  intelligence  you  produce  to 
accomplish  their  missions  are  customers  for  your  AOR  and  should  be  identified  as  part  of  the 
intelligence  issue  process.  For  a  self-initiated  issue,  you  want  to  identity  those  that  can  use  the 
intelligence  assessment  that  you  will  eventually  produce.  These  customers  may  be  inside  or 
outside  of  intelligence  organizations.  For  external  issues,  the  customer  is  the  person  or 
organization  that  submitted  the  question  or  issue  to  you  for  analysis. 

What  is  the  result  of  developing  an  intelligence  issue  statement? 

Developing  an  intelligence  issue  statement  as  described  in  this  FAC  section  is  designed  to  create 
a  clear,  succinct  statement  that  captures  the  essence  of  the  intelligence  issue.  The  intelligence 
issue  statement  is  a  reflection  of  how  the  issue  is  understood  by  both  the  customer  and  the 
analyst.  It  contributes  to  a  satisfactory  outcome  of  the  analysis  activity. 

What  are  the  benefits  of  exploring  an  issue  before  I  begin  any  other  intelligence  analysis 
activities? 

How  you  understand  an  issue  affects  all  the  other  decisions  you  make  in  planning,  research,  and 
analysis.  For  example,  it  determines  the  information  you  will  seek,  those  you  will  collaborate 
with,  and  the  areas  where  you’ll  focus  your  analytic  effort.  The  better  your  understanding,  the 
more  effective  you’ll  be  as  you  do  your  analysis  activities. 

Developing  an  intelligence  issue  statement  helps  you  establish  a  better  relationship  with 
customers: 

•  Working  to  understand  a  customer’s  question  demonstrates  your  interest  in  his  or 
her  goals,  a  willingness  to  help  customers  achieve  those  goals,  and  your  desire  for  a 
collaborative  relationship 

•  The  guides  can  help  you  structure  your  interaction  with  the  customer  thereby 
creating  a  more  focused  and  professional  discussion 

•  The  process  results  in  a  better  understanding  of  the  intended  outcome  of  the 
intelligence  effort,  which  helps  you  manage  both  customer  expectations  and  your 
analysis 

You  may  save  time  in  the  long  run.  Studies  of  similar  kinds  of  upfront  work  in  related 
disciplines  show  huge  payoffs  for  taking  the  time  to  understand  the  customer’s  needs.  For 
example,  each  hour  invested  in  understanding  customer’s  needs  in  software  development  has 
been  estimated  to  save  anywhere  from  30-200  hours  of  development  time  once  coding  has 
begun.  Similar  studies  have  not  been  done  for  intelligence  analysis,  but  it  is  not  too  great  a  leap 
to  see  that  a  well-developed  intelligence  issue  statement  can  prevent  wasted  intelligence 
resources  and  effort. 

Even  if  you’re  experienced,  the  guides  in  this  section  can  help  you  corroborate  your  work  and 
assure  that  key  issues  and  deliverables  have  been  appropriately  considered  by  both  you  and  the 
customer. 
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What  are  the  benefits  for  the  customer? 

The  primary  benefit  to  customers  is  that  thoughtfully  and  thoroughly  exploring  a  question  or 
issue  the  customer  is  interested  in  increases  the  probability  that  the  assessment  provided  will  be 
relevant  to  customers’  issues  and  concerns.  Exploring  an  issue  with  customer  participation 
provides  heightened  awareness  of  an  issue  and  its  related  considerations.  If  the  intelligence 
analysis  effort  is  unsatisfactory,  an  explicit  issue  statement  is  a  starting  point  for  re-interpretation 
of  the  issue  or  for  discussion  about  how  the  issue  has  evolved.  Finally,  the  intelligence  issue 
statement  created  as  a  result  of  the  process  can  serve  as  an  agreement  between  customer  and 
analyst.  The  existence  of  the  intelligence  issue  statement  helps  all  parties  manage  their 
expectations  regarding  the  intelligence  effort. 

How  can  I  benefit  from  developing  an  intelligence  issue  statement? 

Developing  an  intelligence  issue  statement  helps  you  get  oriented  to  the  intelligence  issue  right 
away.  The  guides  discussed  in  this  section  provide  a  starting  point  so  that  you  do  not  have  to 
begin  from  scratch  each  time  you  receive  or  initiate  an  intelligence  issue.  You  may  have  to  add 
questions  to  the  guides  we  supply  to  you,  but  you  will  at  least  have  a  basic  set  of  questions 
developed  by  experienced  intelligence  analysts  to  help  you  begin.  Most  importantly,  using  a 
consistent  process  for  developing  an  issue  statement  increases  the  probability  of  eventual 
customer  satisfaction,  resulting  in  a  customer  who  values  your  work.  It  also  increases  the 
probability  that  you’ll  do  a  focused,  thorough  analysis. 

What  is  the  relationship  between  developing  an  intelligence  issue  statement  and  other  essential 
elements  of  the  intelligence  analysis  process? 

Euripides  once  said:  “A  bad  beginning  makes  a  bad  ending”.  The  more  completely  you 
understand  the  issue,  the  better  your  decisions  will  be  throughout  your  analysis.  As  covered  in 
prior  FAC  sections,  your  purpose  for  the  analysis  establishes  the  basis  for  assigning  meaning  to 
information.  For  example,  your  intelligence  issue  statement  will  shape  your  development  of  an 
information  acquisition  plan,  selection  of  relevant  items  during  your  information  searches,  and 
ultimately  your  assignment  of  meaning. 

What  if  I  am  the  one  posing  the  question  and  don ’t  yet  have  a  customer  ( i.  e. ,  a  self-initiated 
issue)? 

If  you  are  working  on  a  self-initiated  intelligence  effort,  you  still  need  to  explore  the  question  so 
that  you  make  the  best  use  of  your  time  and  resources.  You  can  use  the  guides  provided  in  this 
FAC  section  to  ask  yourself  questions  that  help  set  the  direction  of  your  inquiry  and  stimulate 
your  thinking  about  who  the  intended  audience  is  for  the  ultimate  product.  When  you  have 
identified  intended  audiences,  i.e.,  customers,  you  can  initiate  contact  to  discuss  the  issue. 

What  if  I’m  contributing  analysis  to  another  analyst’s  intelligence  issue? 

If  you  are  providing  contributing  analysis  to  another  analyst’s  intelligence  issue,  you  can  also  use 
the  guides  to  ask  yourself  questions  and  interact  with  the  other  analyst  to  understand  what  is 
needed.  Your  mutual  collaboration  on  exploring  various  aspects  of  the  intelligence  issue  will 
result  in  a  more  focused,  thorough  analysis  for  the  ultimate  customer. 
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Don  7  intelligence  analysts  already  have  a  process  for  exploring  intelligence  issues? 

We’d  be  willing  to  bet  lunch  that  many  do,  but  they  are  processes  that  they’ve  developed  for 
their  own  use.  Unfortunately,  the  intelligence  literature  is  mostly  silent  on  this  issue.  Many 
authors  acknowledge  that  some  problem  exploration  and  definition  has  to  take  place,  but  they 
don’t  provide  many  details.  We  searched  high  and  low  and  couldn’t  find  any  how-to  guides, 
even  in  fields  outside  of  intelligence  analysis.  Almost  all  published  intelligence  literature 
focuses  on  answering  an  intelligence  question,  not  on  how  the  question  or  issue  might  best  be 
formulated.  Exploring  the  intelligence  issue  is  a  form  of  analysis  itself  and  one  that  has  been 
largely  ignored  in  the  intelligence  analysis  literature.  We  also  emphasize  capturing  your 
thoughts  in  writing  and  developing  a  written  intelligence  issue  statement  because  the  writing 
benefits  your  thinking,  as  pointed  out  in  the  previous  material  on  “Writing  as  Thinking.” 
Consequently,  a  written  statement  may  shape  your  analysis  in  a  substantially  different  way  than 
if  you  rely  solely  on  mental  exploration  of  the  issue. 

4.2  Framing 

The  intelligence  issue  statement  process  we  describe  in  this  FAC  section  relies  on  a  technique 
known  as  framing.  “Framing  is  the  process  by  which  people  consciously  or  unconsciously 
impose  an  assumed  structure  on  a  situation  by  selecting  relevant  features:  what  is  important  and 
what  is  less  important.”71  Framing,  in  the  context  of  the  FAC,  is  a  conscious  form  of  inquiry  that 
helps  you  develop  a  structure  for  the  issue.  Using  framing  helps  you  identify  emergent 
conditions,  define  the  scope  of  an  issue,  and  develop  a  greater  understanding  of  an  issue’s 
component  parts.  While  you  are  framing  an  issue  you  may  also  identify  underlying  components 
of  the  issue  that  had  been  hidden  from  view  initially. 

Framing  is  also  an  iterative  and  collaborative  activity.  As  the  intelligence  analyst,  you  lead  the 
framing  effort,  but  framing  also  relies  on  your  subject  matter  expertise,  information  gained  from 
the  customer,  other  analysts,  and  any  research  you  might  conduct  while  developing  the 
intelligence  issue  statement  to  attain  a  richer  understanding  of  the  issue. 

By  applying  the  framing  technique  described  in  this  FAC  section,  you  will  have  a  robust 
understanding  of  the  issue  before  you  begin  your  analysis  of  it.  And,  the  intelligence  issue 
statement  you  create  as  a  result  of  framing  will  be  an  explicit  expression  of  what  the  customer 
needs  in  order  to  achieve  his  or  her  particular  goals. 

4.2.1.  Framing  for  Intelligence  Analysis 

The  framing  technique  created  for  intelligence  analysis  relies  on  two  views  that  can  be  applied  to 
intelligence  issues: 

•  Customer 

•  Target 

The  customer  view  considers  aspects  of  the  person  or  organization  who  is  the  intended  audience 
for  the  intelligence  issue.  It  attempts  to  provide  more  understanding  of  the  basic  questions  “Who 
is  posing  the  intelligence  issue  and  how  are  they  planning  to  use  the  assessment?”  or  “Who  could 
use  the  intelligence  I’ll  produce  to  accomplish  their  mission?”  Inquiry  within  the  customer  view 
attempts  to  understand  the  customer’s  organization,  objectives,  roles,  missions,  priorities, 


71  Hey,  J.  (2005).  Effective  Framing  in  Design,  pp.  2 
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breadth  and  detail  of  results  needed,  and  the  circumstances  that  led  to  the  question.  If  it  is  a  self- 
initiated  or  AOR  issue,  then  you,  your  analytic  colleagues,  and  your  management  still  need  to 
consider  customer  interests  and  how  customers  might  use  the  results  of  your  work. 

A  target  view  considers  aspects  of  the  subject  of  interest  for  your  analysis.  The  target  view  does 
not  pose  questions  in  an  attempt  to  understand  everything  there  is  to  know  about  the  target.  You 
will  discover  what  you  need  to  know  as  you  proceed  through  the  FAC  and  do  your  analysis. 
Instead,  questions  in  the  target  view  are  an  attempt  to  understand  what  the  actual  or  prospective 
customer  needs  to  know  about  the  target  from  the  perspective  of  a  customer.  This  understanding 
of  what  the  customer  needs  to  know  will  shape  your  analysis.  It  will  help  you  understand  what 
you  need  to  know  and  the  information  you’ll  need  to  acquire  to  address  the  intelligence  issue  (as 
described  in  later  sections). 

We  have  organized  the  FAC  so  that  inquiry  within  the  customer  and  target  views  leads  to  the 
intelligence  issue  statement. 

JUST  A  NOTE  BEFORE  YOU  BEGIN  TO  FRAME  A  QUESTION 

Inquiry  within  the  customer  and  target  views  depends  on  subject  matter  knowledge.  For  the 
purposes  of  the  FAC,  we  assume  that  intelligence  analysts  have  at  least  a  conceptual 
understanding  of  the  subject  area  under  consideration.  If  you  have  some  background  in  a 
subject  area,  you  may  want  to  explore  the  topic  further  after  you  receive  the  customer’s 
question  so  that  you  are  prepared  to  elicit  more  information  from  the  customer  or  your 
colleagues.  If  you  do  not  have  at  least  a  background  understanding  of  the  topic,  you  may  want 
to  consider  activities  that  would  improve  your  knowledge  of  the  area  under  study  before  you 
begin  to  frame  the  question.  If  you  lack  knowledge  of  the  subject  matter,  you  will  find  it 
difficult  to  frame  a  question  effectively. 


Why  does  the  approach  taken  in  the  FAC  emphasize  framing  and  developing  an  intelligence 
issue  statement? 

One  interesting  finding  in  our  research  was  the  contrast  in  behavior  between  novices  and  experts 
when  presented  with  a  problem  to  solve.  Novices  base  interpretation  of  problems  and  their 
approach  to  solving  them  on  the  literal  features  of  the  initial  problem  statement.  Novices  are  less 
likely  to  notice  features  and  patterns  in  problem  solving  settings.  Experts,  on  the  other  hand, 
spend  time  framing  and  exploring  an  issue  and  identifying  constraints  before  they  attempt  to 
solve  problems.  We  like  to  think  that  an  emphasis  on  developing  an  issue  statement,  backed  up 
with  the  guides  to  help  you  frame  issues  and  expand  your  subject  matter  knowledge,  will  allow 
you  to  produce  the  results  of  an  expert,  not  a  novice. 

Does  the  use  of  framing  result  in  a  hypothesis? 

No,  that  is  not  the  objective  of  framing.  The  framing  technique  we  discuss  is  intended  to 
establish  the  boundaries  of  an  intelligence  issue  and  create  a  mutual  understanding  of  the 
intelligence  issue.  As  a  result,  the  direction  and  focus  of  your  intelligence  analysis  efforts  are  as 
clear  as  they  can  be.  Framing  gives  a  broad  fonn  to  an  issue  and  states  the  goals  and  objectives 
of  the  analysis.  At  the  same  time,  framing  deliberately  limits  the  scope  of  the  inquiry;  it  tells  you 
where  you  will  not  go  while  trying  to  address  the  intelligence  issue. 
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4.3  Guides  for  Framing 

Since  framing  relies  on  inquiry,  asking  questions  is  an  important  part  of  understanding  the 
customer  and  the  target.  To  help  you  frame  an  issue  we  have  created  framing  guides  which 
contain  a  variety  of  questions  about  the  customer  and  the  target.  The  questions  on  the  guides 
were  developed  with  the  help  of  experienced  intelligence  analysts  and  are  based  on  issues  that 
recur  in  the  intelligence  literature.  The  questions  are  intended  to  guide  research,  discussions,  and 
your  thinking  as  you  develop  an  intelligence  issue  statement. 

It  is  important  to  understand  that  the  framing  guides  do  not  contain  every  question  you  might 
need  to  ask  on  an  issue.  Rather,  they  are  intended  to  help  you  capture  the  core  aspects  that  may 
need  to  be  addressed.  We  recognize  that  each  intelligence  issue  contains  unique  aspects  that 
need  to  be  addressed;  we  expect  that  you  will  add  your  own  questions  to  the  guides  before  and  as 
you  use  them.  Both  guides  will  likely  require  multiple  iterations  as  you  refine  your  thinking 
after  discussions  with  the  customer  and  other  analysts. 

The  guide  for  the  customer  view  is  in  Supplement  1  to  this  section;  the  target  view  guide  is  in 
Supplement  2.  If  the  issue  is  an  AOR  or  self-initiated  issue,  use  the  customer  view  guide  to 
stimulate  your  thinking  about  how  prospective  customers  could  use  the  analysis  and  why  it 
would  be  important  to  them.  If  you  are  contributing  analysis  to  another  analyst,  use  the  customer 
view  guide  to  understand  how  that  analyst  will  use  your  analytic  contribution. 

Next  we’ll  discuss  each  of  the  phases  of  developing  an  intelligence  issue  statement.  We’ll 
integrate  the  concepts  and  guides  we’ve  just  discussed  into  the  process  and  give  you  other  hints 
for  developing  an  intelligence  issue  statement.  The  activities  and  guides  we  describe  are 
designed  to  increase  your  cognitive  engagement  with  the  issue  by: 

•  Following  a  process  for  developing  an  intelligence  issue  statement 

•  Using  guides  for  framing,  including  modifying  those  guides  to  suit  the  current  issue 

•  Aiding  you  in  becoming  immersed  in  the  issue 

•  Stimulating  your  thinking  and  learning  by  writing 

•  Engaging  with  the  customer  and  colleagues  to  corroborate  your  understanding  of 
the  issue 

4.4  Developing  an  Intelligence  Issue  Statement 

The  question  grows  out  of  an  intense  interest  in  a  particular  problem  or  theme.  The 
researcher's  excitement  and  curiosity  inspire  the  search;  associations  multiply  as  personal 
experiences  bring  the  core  of  the  problem  into  focus.  As  the  fullness  of  the  theme  emerges, 
strands  and  tangents  of  it  may  complicate  an  articulation  of  a  manageable  and  specific 
question.  Yet  this  process  of  allowing  all  aspects  to  come  into  awareness  is  essential  to  the 
eventual  formulation  of  a  clear  question.  7~ 

Developing  an  issue  statement  consists  of  the  following  activities: 

•  Receive  the  initial  intelligence  issue  (i.e.,  be  assigned  an  external  issue  or  new  AOR 
or  develop  a  self-initiated  issue) 

•  Appraise  the  issue 

72  Moustakas,  C.E.  Heuristic  Research:  Design,  Methodology’,  and  Applications,  Sage  Publications,  Newbury  Park, 
CA,  1990,  p.  41. 
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•  Conduct  research  to  gain  sufficient  background  for  framing  the  issue 

•  Write  the  intelligence  issue  statement 

•  Corroborate  the  intelligence  issue  statement 

4.4.1.  Receive  the  Initial  Intelligence  Issue 

An  intelligence  issue  often  comes  to  the  attention  of  an  intelligence  analyst  in  the  fonn  of  a 
question.  Something  new  or  different  in  the  world  is  often  the  source  of  a  question.  Whether  the 
question  arises  during  a  conversation,  arrives  through  a  formal  question  submission  from  a 
customer,  or  emerges  while  you  are  doing  research  on  another  topic,  the  presence  of  a  question  is 
a  kickoff  event  for  developing  an  intelligence  issue  statement.  In  the  case  of  an  AOR  issue,  you 
are  assigned  to  understand  a  new  AOR  and  there  are  many  questions  associated  with  attaining 
that  understanding. 

4.4.2.  Appraise  the  Issue 

Before  you  undertake  any  activities  to  frame  an  issue,  you  need  to  consider  the  issue  in  a 
comprehensive  way.  Appraising  the  issue  consists  of  two  main  areas  of  activity:  an  assessment 
of  your  subject  matter  knowledge  related  to  the  issue  and  preparing  to  conduct  the  research  that 
helps  you  frame  the  issue. 

To  assess  your  subject  matter  knowledge,  ask  yourself  the  following  questions: 

Does  the  issue  make  sense  to  you?  What  do  you  currently  know  about  the  topic?  Is  it 
directly  related  to  your  area  of  expertise?  Do  you  need  to  do  more  research  to  understand 
the  issue  well  enough  to  accomplish  the  activities  associated  with  framing  it,  such  as 
discussing  it  with  customers,  other  analysts,  and  management?  Do  you  know  anything 
about  the  requestor  or  his  or  her  organization?  Have  you  worked  with  him  or  her  in  the 
past?  Are  knowledgeable  colleagues  available  for  consultation  on  the  subject  areas  covered 
in  the  intelligence  question?  Do  you  need  to  engage  in  other  activities  to  explore 
information  concerning  the  target? 

To  prepare  for  the  research  that  helps  you  frame  an  issue,  you  may  need  to  do  any  or  all  of  the 
following: 

•  Review  the  customer  and  target  guides  and  add  or  delete  questions  as  necessary 

•  Decide  who  you  will  need  to  talk  with  about  the  issue 

•  Note  that  discussions  do  not  necessarily  need  to  be  face-to-face.  Contact  can  be 
through  a  variety  of  interactive  approaches 

•  For  external  and  self-initiated  issues,  check  existing  intelligence  resources  to 
determine  whether  the  issue  has  been  posed  previously.  Does  existing  intelligence 
address  the  issue,  in  whole  or  in  part?  For  AOR  issues,  identify  what  organizations 
and  analysts  do  analysis  on  related  or  contributing  AORs. 

Hints: 

•  After  you  have  appraised  the  issue  and  modified  the  customer  and  target  guides, 
answer  the  questions  on  the  guides  yourself.  This  helps  you  think  about  the  issue 
systematically  and  may  provide  more  insight  into  the  intelligence  issue.  As  a  result, 
you  may  revise  the  guides,  change  your  concept  of  the  question,  or  decide  to 
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research  a  little  further.  In  addition,  you  will  have  a  basis  for  the  discussions  with 
others  that  will  take  place  in  the  next  phase  of  developing  an  intelligence  issue 
statement. 

4.4.3.  Conduct  Research  for  Framing  the  Issue 

During  this  phase  you  will: 

•  Conduct  discussions  with  actual  or  prospective  customers  or  other  analysts  and 
managers  using  the  framing  guides 

•  Review  the  results  of  the  discussions 

•  Enhance  the  information  you  obtained  during  the  discussions  with  background 
infonnation 

•  Conduct  any  other  types  of  research  needed  to  fill  in  the  gaps  revealed  by  work  on 
the  framing  guides  or  discussions 

•  Conduct  follow  up  activities  such  as  follow-on  discussions  or  calls  if  needed 

4.4.4.  Write  the  Intelligence  Issue  Statement 

The  written  intelligence  issue  statement  provides  direction  for  the  intelligence  effort.  It  reflects 
your  understanding  of  what  you  intend  to  explore  in  the  intelligence  effort  and  why  the  effort  is 
being  undertaken.  In  addition,  the  process  of  writing  the  intelligence  issue  statement  provides  an 
opportunity  for  you  to  deepen  your  understanding  of  the  issue  and  the  customer.  Constructing  an 
intelligence  issue  statement  synthesizes  the  knowledge  you  gained  while  framing  the  issue  from 
the  customer  and  target  views. 

We  recommend  that  an  intelligence  issue  statement  contain  the  following: 

•  A  Goals  and  Objectives  section  describing  what  the  intelligence  effort  is  to 
accomplish.  It  should  identify  the: 

>  Actual  or  prospective  customers  interested  in  the  effort,  by  name, 
organization,  or  function 

>  Current  or  future  events  driving  the  intelligence  issue 

>  Target  of  the  effort 

>  What  the  customer  intends  to  do  with  the  results  of  the  intelligence  effort  or 
how  a  prospective  customer  might  use  the  analysis  to  support  their  mission 

•  A  Focus  section  describing  what  the  customer  needs  to  know  about  the  target.  It 
defines  the  scope  of  the  intelligence  effort  by  describing  those  aspects  that  the 
customer  is  interested  in  knowing  and  the  coverage  of  the  target. 

•  A  Special  Considerations  section  with  notes,  guidelines,  or  other  factors  that  might 
be  related  to  the  intelligence  issue  statement.  This  section  may  also  be  used  to 
elaborate  those  issues  that  will  not  be  considered  in  this  intelligence  effort. 

•  A  Bibliography  of  existing  intelligence  products  that  are  related  to  the  issue.  The 
bibliography  should  include  those  products  that  seem  to  address  some  or  all  aspects 
of  the  intelligence  issue.  The  purpose  of  this  bibliography  is  to  promote  discussion 
with  the  customer  regarding  existing  products  and  ensure  you  are  aware  of  related 
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intelligence  production.  It  should  be  marked  as  a  preliminary  or  working  body  of 
products,  not  as  a  final  reference  list. 

An  example  of  using  the  customer  and  target  view  guides  to  develop  an  intelligence  issue 
statement  is  shown  in  subsection  4.15  “The  Case  of  General  Alpha  -  Facing  Insurgency.” 

Hints: 

•  Write  the  intelligence  issue  statement  from  the  perspective  of  the  customer. 

•  Be  as  explicit  as  possible  without  overly  constraining  the  issue. 

•  Consider  an  evolution  in  your  understanding  of  the  issue  a  positive  development. 
Both  framing  the  intelligence  issue  and  writing  the  intelligence  issue  statement  are 
activities  that  are  designed  to  improve  your  understanding  of  an  issue.  You  should 
expect  to  readdress  the  issue  as  your  understanding  evolves. 

4.4.5.  Corroborating  the  Intelligence  Issue  Statement 

After  you  have  written  the  intelligence  issue  statement,  it  is  important  to  corroborate  it. 
Corroboration  is  intended  to  give  you,  the  customer  (if  applicable),  and  your  colleagues  the 
opportunity  to  confirm  that  the  way  you  have  framed  the  question  matches  the  objectives  to  be 
accomplished.  Corroboration  is  also  another  means  of  solidifying  your  understanding  of  the 
issue. 

Corroboration  consists  of  a  discussion,  either  in-person  or  electronically,  of  the  intelligence  issue 
statement.  This  is  a  deliberate  opportunity  for  you  to  discuss  any  comments  or  questions  and 
achieve  a  shared  understanding  of  the  intelligence  issue  statement  before  you  proceed  with  your 
analysis. 

If  it  is  an  external  issue,  the  customer  is  your  primary  resource  for  corroborating  your 
intelligence  issue  statement.  To  prepare  the  customer  (or  others)  for  their  role  in  corroborating 
the  intelligence  issue  statement,  you  should: 

•  Select  the  medium  you  intend  to  use  for  the  corroboration.  Will  the  corroboration 
occur  face-to-face,  through  email,  through  video  teleconferencing,  or  some  other 
medium? 

•  Make  the  appropriate  arrangements  and  appointments  for  the  medium  selected  in 
preparation  for  the  discussion  of  the  intelligence  issue  statement. 

•  Be  sure  to  make  the  purpose  of  the  meeting  clear  when  you  arrange  the  discussion 
of  the  intelligence  issue  statement. 

•  Send  the  intelligence  issue  statement  to  the  customer  or  others  in  advance  of  the 
discussion  session. 

•  Conduct  the  discussion  about  the  intelligence  issue  statement. 

•  Make  any  changes  needed  to  the  intelligence  issue  statement. 

Hints: 

•  If  intelligence  products  exist  that  seem  to  be  relevant  to  the  issue,  review  these  as 
part  of  the  discussion  of  the  intelligence  issue  statement.  Obtain  feedback  regarding 
the  existing  intelligence  products.  If  the  customer  or  others  think  the  existing 
intelligence  does  not  fully  address  the  issue,  try  to  determine  how  the  existing 
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products  are  inadequate:  is  it  the  product’s  scope,  detail,  content,  or  some  other 
issue?  The  feedback  on  existing  intelligence  products  may  provide  you  with 
important  insights  regarding  the  current  effort. 

•  To  make  corroboration  easier  to  understand,  we  discuss  it  in  this  FAC  section  as  if 
you  would  conduct  a  review  of  the  intelligence  issue  statement  only  one  time.  We 
would  like  to  emphasize  that  it  is  likely  that  you  will  repeat  the  corroboration  more 
than  once  in  order  to  reach  a  satisfactory  agreement  regarding  the  intelligence  issue 
statement.  Your  initial  corroboration  discussions  will  likely  result  in  new  insights, 
direction,  or  dimensions  for  study.  You  should  repeat  the  corroboration  activities 
until  both  you  and  the  customer  or  other  colleagues  are  satisfied  with  the 
intelligence  issue  statement. 

What  can  you  do  if  a  customer  cannot  participate  in  corroborating  the  statement  you  created? 
You  may  be  able  to  arrange  a  meeting  with  an  intermediary  for  the  customer,  such  as  the 
customer’s  staff,  colleagues  with  first-hand  knowledge  of  the  customer,  or  members  of 
intelligence  organizations  that  represent  the  customer.  For  example,  if  your  customer  is  the  Air 
Force  Chief  of  Staff,  you  may  need  to  meet  with  members  of  the  AF/A2  staff,  those  in  the  Air 
Force  Intelligence  Analysis  Agency  (AFIAA),  or  a  special  advisor  to  the  Chief.  The  process 
described  below  for  identifying  reviewers  for  AOR  or  self-initiated  issues  may  also  be  useful  to 
you  if  the  customer  is  not  available. 

If  it  is  an  AOR  or  self-initiated  issue,  you  still  need  to  corroborate  the  intelligence  issue 
statement.  For  AOR  issue  and  self-initiated  issues,  instead  of  working  with  an  actual  customer  to 
review  the  intelligence  issue  statement,  you  can  consider  selecting  reviewers  from  your 
organization.  Reviewers  should  be  knowledgeable  about  the  target  under  consideration  and 
willing  to  serve  as  “customers”  to  corroborate  the  intelligence  issue  statement  you  have  created. 
For  self-initiated  issues  you  can  also  discuss  it  with  prospective  customers  that  you  have 
identified.  If  you  are  contributing  analysis  to  another  analyst,  you  can  work  with  that  analyst 
(and  possibly  other  contributing  analysts)  to  corroborate  the  intelligence  issue  statement  for  your 
contributing  analysis. 


A  NOTE  ABOUT  CORROBORATION 

One  outcome  of  developing  the  intelligence  issue  statement  is  worthy  of  special 
emphasis.  Since  we  recommend  that  you  research  existing  intelligence  products  related 
to  the  customer’s  issue,  it  is  possible  for  external  and  self-initiated  issues  that  you  will 
retrieve  a  completed  intelligence  product  that  addresses  the  customer’s  issue.  Be  sure 
to  present  and  discuss  existing  intelligence  during  the  review  of  the  intelligence  issue 
statement.  If,  during  the  review,  the  customer  asserts  that  the  existing  intelligence 
addresses  his  or  her  issue,  you  can  consider  your  work  on  the  intelligence  issue 
complete. 
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4.5  Developing  Intelligence  Issue  Statements  in  Real  Life 

We  have  discussed  the  process  in  a  step-by-step  sequence  to  help  you  understand  each  of  the 
activities  more  easily.  When  you  actually  put  this  process  into  practice,  you  will  likely  find 
yourself  looping  back  through  the  process  several  times  as  you  develop  the  intelligence  issue 
statement.  Looping  back  is  completely  normal  and  is  most  likely  a  sign  that  as  you  leam  more 
about  the  intelligence  question  you  will  refine  the  intelligence  issue  statement. 

You  may  also  find  that  you  need  to  refine  or  re-frame  the  intelligence  issue  as  you  encounter 
new  developments  or  information  in  later  phases  of  the  intelligence  analysis  process.  Refining 
the  intelligence  issue  statement  as  you  encounter  new  information  is  also  completely  nonnal  and 
is  a  positive  sign. 

4.6  In  Conclusion 

By  developing  intelligence  issue  statements,  you  thoroughly  consider  the  intelligence  issue  from 
the  customer  and  target  frame  perspectives.  Through  research,  discussions,  and  your  subject 
matter  knowledge,  you  create  a  statement  of  the  intelligence  issue  and  corroborate  it  with  the 
customer  who  asked  the  question  or  with  other  knowledgeable  colleagues. 

Once  the  customer  or  your  colleagues  agree  with  the  intelligence  issue  statement  that  you’ve 
created,  you  are  ready  to  begin  planning  the  analysis  effort.  In  the  next  FAC  section,  we’ll 
discuss  how  to  create  an  Analysis  Plan  to  prepare  for  later  phases  of  the  analytic  effort. 

4.7  Using  the  Supplements  and  Example 

In  order  to  illustrate  the  concepts  we  present  in  this  section  of  the  FAC  we  have  provided  guides 
and  a  sample  external  issue  that  uses  the  guides.  Supplements  1,3,  and  5  contain  blank  Customer 
View  Guides  for  external,  self-initiated,  and  AOR  issues.  Supplements  2,  4,  and  6  contain  blank 
Target  View  Guides  for  external,  self-initiated,  and  AOR  issues. 

Subsection  4.14  contains  an  explanation  of  the  sample  issue  we  developed  for  this  section  of  the 
FAC  and  completed  Customer  and  Target  View  Guides,  as  well  as  a  completed  Intelligence 
Issue  Statement.  Please  note  that  we  provide  an  example  for  an  external  issue  only. 
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4.8  Supplement  1:  Customer  View  Guide  for  an  External  Issue 


ITERATION  #:  DATE: 

INITIAL  ISSUE: 

Who  is  the  customer  (organization,  person,  or  role)? 

What  is  the  customer’s  organizational  affiliation?  What  is  the  reporting  hierarchy  for  the 
customer  (customer’s  organization  one  level  up)? 

What  objective  is  the  customer  trying  to  achieve  by  posing  this  issue?  What  is  the  problem  he  or 
she  is  trying  to  solve?  How  will  this  effort  help  the  customer  fulfill  his/her  mission?  How  is  the 
result  of  the  intelligence  effort  going  to  be  used? 

What  are  the  circumstances  that  led  to  the  request  for  this  intelligence  issue  to  be  addressed? 
What  is  the  context  of  the  issue  in  terms  of  government  plans  and  policies? 

Is  the  result  of  the  intelligence  effort  intended  to  support: 

•  Short-term  or  long-term  effort 

•  Tactical  decision  making 

•  Weapons  system  acquisition 

•  Policy  formulation 

•  Support  of  policy  execution 

•  Provocations  to  cause  other  side  to  act 

•  Operational 

•  Planning 

>  Long  range 

>  Strategic 

>  Operational 

>  Tactical 

What  is  the  customer’s  time  frame  for  the  product? 

What  is  the  customer’s  time  frame  for  his  or  her  decision? 

What  form  should  the  intelligence  assessment  take? 

What  is  the  breadth  and  detail  desired  in  the  result? 
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4.9  Supplement  2:  Target  View  Guide  for  an  External  Issue 


ITERATION  #:  DATE: 

INITIAL  ISSUE: 

What  does  the  customer  want  to  know  about  the  target,  what  is  their  focus,  and  what  are  their 
priorities?  Why  is  the  issue  an  issue?  Why  is  it  important  to  address  this  issue? 

Is  the  situation  of  interest  to  the  customer  an  emerging  condition?  Is  the  situation  recent, 
unexpected,  new? 

Is  the  issue  surrounding  the  target  a  “puzzle  or  a  mystery”?  Whether  the  customer  wants  to 
know  about  a  puzzle  or  mystery  is  important  to  understand  as  you  engage  with  the  customer. 

•  A  puzzle  is  knowable,  but  not  yet  by  us 

•  A  mystery  can  be  known  by  no  one,  because  it  hasn’t  yet  been  decided 

Consider  the  following  to  think  about  what  you  know  and  what  is  unknown  about  the  customer’ 
primary  interests  in  the  target. 

•  Intentions,  Motivations 

•  Military 

•  Diplomatic,  political,  legal 

•  Economic 

•  Cultural,  religious,  sociological,  historical 

•  Information,  media,  propaganda 

•  Infrastructure,  telecommunications,  transportation 

•  Weather 

•  Medical 

•  Criminal 

•  Governmental  structure,  plans,  policies 

•  Commercial,  Business 

•  Scientific/technical 

What  is  the  time  period  that  the  customer  wants  to  know  about?  E.g.,  historical,  current,  future 
(if  so,  how  far  in  the  future) 

Is  a  particular  geography  involved? 

Is  the  political  construct  of  interest  to  the  customer: 

•  Traditional 

•  Transnational 

•  Global 

•  Nation-state 

•  Non- state  actor 
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•  Single  actor 

•  Multiple  actors 

Is  the  customer  interested  in  specific  regional  and  entity  dynamics? 

Is  the  customer  interested  in  the  relationships  this  entity  might  have  with  other  countries  or  other 
entities? 

If  the  target  is  a  person,  what  is  of  interest  to  the  customer: 

•  What  are  the  goals  of  that  person? 

•  Is  the  study  to  be  a  biography? 

•  Is  the  objective  of  the  customer  to  predict  the  future  actions  of  that  person? 

•  Is  the  person  involved  with  a  particular  institution  or  organization? 

Is  a  specific  technology  part  of  what  the  customer  wants  to  know  about  the  issue?  Are  patents 
potentially  of  interest?  If  so,  what  aspects  of  the  technology  is  the  customer  interested  in. 

If  the  target  is  a  system,  which  of  the  systems’  capabilities  are  of  interest  to  the  customer? 

If  the  target  is  a  process,  what  aspects  of  the  process  are  of  interest  to  the  customer? 

If  the  target  is  an  organization,  what  is  the  organizational  form?  Is  the  customer  interested  in 
the  nature  of  the  political  and  economic  system  that  the  organization  works  within? 

•  Non  Governmental  Organization  (NGO) 

•  Government  owned  or  controlled  business 

•  Private  business 
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4.10  Supplement  3:  Customer  View  Guide  for  a  Self-Initiated  Issue 


ITERATION  #:  DATE: 

INITIAL  ISSUE: 

What  are  the  circumstances  that  led  to  you  initiating  this  issue  for  analysis?  Why  is  it 
important  that  this  analysis  be  done? 

Who  would  care  that  this  issue  be  analyzed?  Who  is  the  intended  audience  of  analysis  of  this 
issue,  i.e.,  who  are  the  prospective  customers  (organization,  person,  or  role)? 

What  is  the  prospective  customer’ s(s’)  organizational  affiliation?  What  is  the  reporting 
hierarchy  for  the  customer  [customer’s(s’)  organization  one  level  up]? 

Why  would  prospective  customers  care  about  this  issue?  How  could  prospective  customers  use 
the  assessment  that  will  be  produced  as  a  result  of  this  analysis?  What  objectives  could  they 
achieve  by  knowing  this  information?  What  problem  could  they  solve?  How  would  it  help  the 
customer  fulfill  their  mission? 

What  is  the  context  of  the  issue  in  terms  of  government  plans  and  policies? 

Is  the  result  of  the  intelligence  effort  intended  to  support: 

•  Short-term  or  long-term  effort 

•  Tactical  decision  making 

•  Weapons  system  acquisition 

•  Policy  formulation 

•  Support  of  policy  execution 

•  Provocations  to  cause  other  side  to  act 

•  Operational 

•  Planning 

>  Long  range 

>  Strategic 

>  Operational 

>  Tactical 

When  should  the  analysis  of  the  issue  be  done?  What  is  the  prospective  customer’s  time  frame 
for  the  product? 

What  is  the  time  frame  for  prospective  customers  to  use? 

What  form  should  the  intelligence  assessment  take? 

What  is  the  breadth  and  detail  desired  in  the  result? 
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4.11.  Supplement  4:  Target  View  Guide  for  a  Self-Initiated  Issue 

ITERATION  #:  DATE: 

INITIAL  ISSUE: 


What  do  you  want  to  know  about  the  target,  what  is  your  focus,  what  are  your  priorities?  Why  is 
the  issue  an  issue?  Why  is  it  important  to  address  this  issue? 

Is  the  situation/issue  of  interest  an  emerging  condition?  Is  it  recent,  unexpected,  new? 

What  would  prospective  customers  want  to  know  about  the  target,  what  would  be  their  focus 
and  priorities? 

Is  the  issue  surrounding  the  target  a  “puzzle  or  a  mystery”?  Whether  your  analysis  is  on  a  puzzle 
or  mystery  is  important  to  understand  as  do  your  analysis  and  engage  with  prospective 
customers. 

•  A  puzzle  is  knowable,  but  not  yet  by  us 

•  A  mystery  can  be  known  by  no  one,  because  it  hasn’t  yet  been  decided 

Consider  the  following  to  think  about  what  you  know  and  what  is  unknown  about  prospective 
customers’  interests  in  the  target. 

•  Intentions,  Motivations 

•  Military 

•  Diplomatic,  political,  legal 

•  Economic 

•  Cultural,  religious,  sociological,  historical 

•  Information,  media,  propaganda 

•  Infrastructure,  telecommunications,  transportation 

•  Weather 

•  Medical 

•  Criminal 

•  Governmental  structure,  plans,  policies 

•  Commercial,  Business 

•  Scientific/technical 

What  is  the  time  period  that  you  want  to  know  about?  What  time  frames  would  be  of  interest  to 
prospective  customers?  E.g.,  historical,  current,  future  (if  so,  how  far  in  the  future) 

Is  a  particular  geography  involved? 

Is  the  political  construct  of  interest: 

•  Traditional 

•  Transnational 

•  Global 

•  Nation-state 
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•  Non- state  actor 

•  Single  actor 

•  Multiple  actors 

Are  you  interested  in  specific  regional  and  entity  dynamics?  What  would  prospective  customers 
be  interested  in? 

Are  you  interested  in  the  relationships  this  entity  might  have  with  other  countries  or  other 
entities?  What  relationships  would  customers  be  interest  in? 

If  the  target  is  a  person,  what  is  of  interest  to  you  and  prospective  customers: 

•  What  are  the  goals  of  that  person? 

•  Is  the  study  to  be  a  biography? 

•  Is  the  objective  of  the  customer  to  predict  the  future  actions  of  that  person? 

•  Is  the  person  involved  with  a  particular  institution  or  organization? 

Is  a  specific  technology  part  of  what  you  and  prospective  customers  want  to  know  about  the 
issue?  Are  patents  potentially  of  interest?  If  so,  what  aspects  of  the  technology  is  the  customer 
interested  in. 

If  the  target  is  a  system,  which  of  the  systems’  capabilities  are  of  interest  to  you  and  prospective 
customers? 

If  the  target  is  a  process,  what  aspects  of  the  process  are  of  interest  to  you  and  prospective 
customers? 

If  the  target  is  an  organization,  what  is  the  organizational  form?  Is  the  nature  of  the  political 
and  economic  system  that  the  organization  works  within  of  interest  to  you  and  prospective 
customers? 

•  NGO 

•  Government  owned  or  controlled  business 

•  Private  business 
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4.12 


Supplement  5:  Customer  View  Guide  for  an  APR  Issue 


ITERATION  #:  DATE: 

INITIAL  ISSUE: 

What  is  the  proposed  scope  of  the  analysis  for  the  AOR  to  which  you’ve  been  assigned?  Why  is 
it  important  that  this  analysis  be  done? 

What  questions  do  you  have  about  the  scope  of  the  AOR? 

What  would  understanding  of  this  AOR  contribute  to  the  knowledge  of  the  intelligence 
community  and  its  ability  to  support  customers? 

Who  would  care  that  this  AOR  be  analyzed?  Who  is  the  intended  audience  of  analysis  of  the 
AOR,  i.e.,  who  are  the  prospective  customers  (organization,  person,  or  role)? 

What  are  the  prospective  customers’  organizational  affiliations? 

Why  would  prospective  customers  care  about  this  AOR?  How  could  prospective  customers  use 
the  assessment  that  will  be  produced  as  a  result  of  this  analysis?  What  objectives  could  they 
achieve  by  knowing  this  information?  What  problems  could  they  solve?  How  would  it  help 
customers  fulfill  their  missions? 

How  do  the  interests  of  various  prospective  customers  vary? 

What  is  the  context  of  the  issue  in  terms  of  government  plans  and  policies? 

Is  the  result  of  analyzing  this  AOR  intended  to  support: 

•  Short-term  or  long-term  effort 

•  Tactical  decision  making 

•  Weapons  system  acquisition 

•  Policy  formulation 

•  Support  of  policy  execution 

•  Provocations  to  cause  other  side  to  act 

•  Operational 

•  Planning 

>  Long  range 

>  Strategic 

>  Operational 

>  Tactical 

When  should  the  analysis  of  the  AOR  be  done? 

What  form  should  intelligence  assessments  for  this  AOR  take? 

Typically  what  is  the  breadth  and  detail  desired  in  assessments  for  the  AOR? 
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4.13  Supplement  6:  Target  View  Guide  for  an  APR  Issue 


ITERATION  #:  DATE: 

INITIAL  ISSUE: 

What  do  you  want  to  know  about  the  AOR,  what  is  your  focus,  what  are  your  priorities?  Why  is 
it  important  to  address  this  AOR? 

What  situation/issue  in  the  AOR  are  emerging  conditions,  recent,  unexpected,  new?  What 
aspects  seem  relatively  static? 

What  would  prospective  customers  want  to  know  about  the  target,  i.e.,  the  AOR,  what  would  be 
their  focus  and  priorities? 

What  are  the  “puzzles  or  a  mysteries”  associated  with  the  AOR?  Whether  your  analysis  is  on  a 
puzzle  or  mystery  is  important  to  understand  as  do  your  analysis  and  engage  with  prospective 
customers. 

•  A  puzzle  is  knowable,  but  not  yet  by  us 

•  A  mystery  can  be  known  by  no  one,  because  it  hasn’t  yet  been  decided 

Consider  the  following  to  think  about  what  you  know  and  what  is  unknown  about  prospective 
customers’  interests  in  the  target,  i.e.,  the  AOR. 

•  Intentions,  Motivations 

•  Military 

•  Diplomatic,  political,  legal 

•  Economic 

•  Cultural,  religious,  sociological,  historical 

•  Information,  media,  propaganda 

•  Infrastructure,  telecommunications,  transportation 

•  Weather 

•  Medical 

•  Criminal 

•  Governmental  structure,  plans,  policies 

•  Commercial,  Business 

•  Scientific/technical 

What  is  the  time  period  that  you  want  to  know  about?  What  time  frames  would  be  of  interest  to 
prospective  customers?  E.g.,  historical,  current,  future  (if  so,  how  far  in  the  future) 

Is  a  particular  geography  involved? 

Is  the  political  construct  of  interest: 

•  Traditional 
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•  Transnational 

•  Global 

•  Nation-state 

•  Non- state  actor 

•  Single  actor 

•  Multiple  actors 

Are  you  interested  in  specific  regional  and  entity  dynamics?  What  would  prospective  customers 
be  interested  in? 

Are  you  interested  in  the  relationships  this  entity  might  have  with  other  countries  or  other 
entities?  What  relationships  would  customers  be  interest  in? 

If  the  target  is  a  person,  what  is  of  interest  to  you  and  prospective  customers: 

•  What  are  the  goals  of  that  person? 

•  Is  the  study  to  be  a  biography? 

•  Is  the  objective  of  the  customer  to  predict  the  future  actions  of  that  person? 

•  Is  the  person  involved  with  a  particular  institution  or  organization? 

Is  a  specific  technology  part  of  what  you  and  prospective  customers  want  to  know  about  the 
issue?  Are  patents  potentially  of  interest?  If  so,  what  aspects  of  the  technology  is  the  customer 
interested  in. 

If  the  target  is  a  system,  which  of  the  systems’  capabilities  are  of  interest  to  you  and  prospective 
customers? 

If  the  target  is  a  process,  what  aspects  of  the  process  are  of  interest  to  you  and  prospective 
customers? 

If  the  target  is  an  organization,  what  is  the  organizational  form?  Is  the  nature  of  the  political 
and  economic  system  that  the  organization  works  within  of  interest  to  you  and  prospective 
customers? 

•  NGO 

•  Government  owned  or  controlled  business 

•  Private  business 
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4.14  Example  Using  an  External  Issue:  Introduction 

This  example  illustrates  two  iterations  on  an  issue  received  from  a  customer,  an  external  issue. 
For  this  example,  what  you  know  is  what  is  in  subsection  4. 14.6  “Case  Study  #1 :  The  Case  of 
General  Alpha:  Facing  Insurgency?”  This  case  study  was  created  by  Frank  J.  Hughes  and  David 
A.  Schum  as  part  of  the  instructional  materials  for  a  course  at  the  Joint  Military  Intelligence 
College  (JMIC). 73 

The  same  exploration  should  be  done  for  AOR  and  self-initiated  issues.  Iteration  1  includes  the 
customer  question  as  received,  what  you  know  before  any  interaction  with  the  customer  about 
the  customer  and  the  target,  and  unknowns  about  the  customer  and  what  the  customer  wants  to 
know  about  the  target.  It  results  in  questions  you  develop  to  research  or  ask  the  customer  or 
other  colleagues.  For  AOR  and  self-initiated  issues,  you  will  need  to  ask  yourself  the  questions 
as  they  relate  to  prospective  customers  (see  Customer  and  Target  View  Guides  for  AOR  and 
self-initiated  issues). 

Iteration  2  incorporates  information  obtained  from  the  customer,  as  well  as  the  results  of  your 
research  during  and  after  Iteration  1 .  In  this  example,  Iteration  2  is  sufficient  to  develop  the 
intelligence  issue  statement  that  you’ll  use  to  begin  your  analysis.  This  is  a  simplified  example, 
with  only  two  iterations.  In  many  cases,  it  will  require  many  more  iterations  to  arrive  at  the  point 
where  analysis  can  proceed  based  on  the  intelligence  issue  statement.  Subsequent  iterations  will 
reflect  your  evolving  understanding  of  the  issue,  with  the  last  iteration  representing  the  issue  that 
you  will  use  to  proceed  with  your  analysis. 

How  you  represent  the  information  in  the  customer  and  target  views  is  up  to  you.  Consequently, 
no  consistent  format  is  used  in  the  example.  But,  you  should  deliberately  explore  what  is 
known  and  what  is  unknown.  The  unknowns  will  be  the  basis  for  you  doing  some  initial 
research  to  prepare  yourself  to  talk  to  the  customer  or  other  colleagues.  The  unknowns  will  also 
form  the  basis  for  questions  you  can  use  to  discuss  the  issue  with  the  customer  and/or  colleagues. 

You  will  likely  find  that  the  questions,  and  even  the  customer  and  target  views,  are  highly 
interdependent  for  many  issues.  Stimulating  your  thinking  and  getting  your  thoughts  down  is 
what’s  important,  not  exactly  where  a  given  piece  of  information  should  be  placed. 

In  addition  to  adding  questions,  you  may  also  want  to  reorder  and/or  regroup  questions  based  on 
the  nature  of  the  issue  and  how  you  think  about  it  to  facilitate  your  consideration  of  the  topics. 


73  Hughes,  F.J.  and  Schum,  D.  A.  “The  Art  and  Science  of  the  Process  of  Intelligence  Analysis.  Case  Study  #1  (The 
Case  of  General  Alpha:  Facing  Insurgency?).  Analysis.”  2006. 
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4.14.1.  Customer  View  Guide  -  External  Issue  -  Iteration  1 


“aspects  of  the  person  or  organization  who  is  the  intended  audience  for  the  intelligence  issue  ” 
“Who  is  posing  the  intelligence  issue  and  how  are  they  planning  to  use  the  assessment?  ” 
(“Developing  Intelligence  Issue  Statements  ”) 


ITERATION#:  1 
DATE:  21  May  09 

INITIAL  ISSUE:  The  Commander  in  Chief  (CINC)  is  concerned  about  this  insurgency 
and  has  asked  the  J2  to  assess  the  likeliness  that  the  insurgent  group  will 
overthrow  General  Alpha. 

Who  is  the  customer  (organization,  person,  or  role)? 

Known:  XXXCOM/J2 

Unknowns:  None 

What  is  the  customer’s  organizational  affiliation?  What  is  the  reporting  hierarchy  for  the 
customer  (customer’s  organization  one  level  up? 

Known:  XXXCOM/J2  (  part  of  XXX  Command) 

Known:  J2  is  the  intelligence  organization  that  works  for  the  CINC.  J2  reports 
directly  to  the  CINC. 

Unknown:  Does  the  CINC  report  directly  to  the  Secretary  of  Defense  (SECDEF)? 
(research) 

What  objective  is  the  customer  trying  to  achieve  by  posing  this  issue?  What  is  the  problem  he  or 
she  is  trying  to  solve?  How  will  this  effort  help  the  customer  fulfill  his/her  mission?  How  is  the 
result  of  the  intelligence  effort  going  to  be  used? 

Knowns:  The  stated  objective  is  that  the  customer  has  been  asked  by  the  CINC  to 
“assess  the  likeliness  that  the  insurgent  group  will  overthrow  General  Alpha."  The 
J2's  role  is  to  provide  intelligence  to  support  the  CINC. 

Unknown:  the  underlying  objective  of  the  CINC.  Talk  with  the  customer  to  see 
what  the  underlying  objective  is.  Is  it  to  advise  on  potential  US  assistance  to 
insurgents  and/or  to  prepare  US  forces  for  various  types  of  actions? 

Other  unknowns  to  discuss  with  the  customer: 

-  Since  the  J2  must  address  this  and  sent  tasking  to  our  organization,  does  the 
J2  want  our  organizations'  independent  assessment  of  the  likelihood  of 
overthrowing  General  Alpha?  Is  the  J2  doing  their  own  assessment  and  asking  for 
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our  view?  Or  is  J2  looking  for  a  community  assessment  of  the  likelihood?  Who 
else  in  the  community  is  looking  at  this  issue? 

-  What  sorts  of  decisions  is  the  CINC  considering,  e.g.,  possible  preparatory 
action  should  it  appear  likely  that  Gen  Alpha  will  be  overthrown?  Assistance  to 
insurgents  or  other  countries'  actions?  Are  they  considering  consequences  if  Gen 
Alpha  is  not  overthrown? 

-  Or,  is  the  CINC  just  curious  about  what's  going  on  and  the  issue  isn't  related  to 
ongoing  planning  for  action?  Are  there  other  reasons  for  the  question? 

-  How  does  the  J2  plan  to  advise  the  CINC?  Will  they  want  analysts  to  provide 
briefings  directly  at  some  point?  Will  the  J2  assimilate  the  information  and 
provide? 

What  are  the  circumstances  that  led  to  the  request  for  the  intelligence  issue  to  be  addressed? 

Known:  Information  from  X  and  Y  (opposition  leaders)  to  Central  Intelligence 
Agency  (CIA)  operatives  that  stage  was  set  for  an  insurgent  operation  against  Gen 
Alpha  sometime  in  the  next  month. 

Unknown:  other  policy  considerations  and  potential  planning  for  U.S.  action.  See 
below.  Discuss  with  customer. 

What  is  the  context  of  the  issue  in  terms  of  government  plans  and  policies? 

Known:  The  insurgency  wants  the  US  to  assist  them  with  military  and  other  aid.  If 
the  US  makes  a  decision  to  do  this,  the  US  must  prepare.  Non-combatant 
evacuation  or  humanitarian  support  may  also  be  needed. 

Unknowns:  existing  US  policies  and  plans  relative  to  Country  Orange.  Do  we  have  a 
policy  to  intervene  in  the  case  of  overthrow?  Do  we  have  plans  in  place  for  such 
actions?  Is  the  issue  related  to  a  need  to  update  such  plans?  Is  non-combatant 
evacuation  or  humanitarian  support  being  planned?  (discuss  with  the  customer) 

Is  the  result  of  the  intelligence  effort  intended  to  support:  (discuss  identified  items  with 
customer) 

•  Short-term  or  long-term  effort:  Question  appears  to  be  short-term,  but 
there  are  long-term  implications;  may  be  longer-term  if  Blues  not  yet 
ready 

•  Tactical  decision  making:  some  implications  here  potentially 

•  Weapons  system  acquisition:  No 

•  Policy  formulation:  implications  here  potentially 

•  Support  of  policy  execution:  implications  here  potentially 
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•  Provocations  to  cause  other  side  to  act:  possibly 

•  Operational:  yes,  potentially 

•  Planning:  yes,  potentially  most  immediate  need  may  be  operational  and 
tactical  planning,  with  knowledge  of  strategic  implications 

>  Long  range 

>  Strategic 

>  Operational 

>  Tactical 

What  is  the  customer’s  time  frame  for  the  product? 

Known:  There  is  information  indicating  insurgents  may  act  sometime  next  month 
(based  on  info  provided  to  CIA  operatives). 

Unknown:  the  reliability  of  this  information.  Do  research  on  the  source.  Discuss 
with  the  customer  what  they  know  about  the  source. 

Unknown:  when  the  customer  wants  the  product.  Nothing  was  included  on  the 
request.  Prior  experience  tells  me  the  customer  probably  will  want  an  answer  in  1 
week  or  less,  maybe  immediately.  Discuss  with  the  J2  staff  and  find  out  how  much 
time  we  have. 

What  is  the  customer’s  time  frame  for  his  or  her  decision? 

Unknown:  what  the  time  f rame  is  for  the  decision  or  what  type  of  decision  is  to  be 
made.  If  the  source  information  is  reliable  on  the  insurgents  acting  sometime  next 
month,  the  decision  must  happen  quickly.  Discuss  with  the  customer. 

What  form  should  the  intelligence  assessment  take? 

Unknown:  Discuss  with  the  customer.  Initial  thoughts  are  concise  narrative 
response,  perhaps  graphics  to  support  text.  Include  description  of  possibilities 
and  likelihoods,  with  conditions  and  uncertainties  relevant  to  customer  made 
explicit;  identify  most  likely  up  front.  Also  discuss  whether  a  briefing  is  needed 
and  interactive  discussions,  e.g.,  depending  on  who  will  present  the  assessment  to 
the  CINC 

What  is  the  breadth  and  detail  desired  in  the  result? 

Unknown:  breadth  and  detail  desired.  Discuss  with  customer.  Initial  thoughts: 
Outline  major  conditions  for  and  against  Gen  Alpha  being  overthrown  in  the  next 
month.  Key  element  is  what  the  insurgents  will  really  do.  Supporting  information 
needed  is:  How  credible  is  the  info  provided  by  X  &  Y?  What  are  the  insurgents' 
capabilities  to  pull  off  overthrowing  Gen  Alpha?  How  capable  is  Gen  Alpha  and  his 
regime  of  countering  the  insurgents?  Can  the  insurgents  pull  off  a  surprise 
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attack?  Does  Gen  Alpha  have  sources  within  the  insurgency?  What  will  be  the  role 
of  the  populace  and  religious  leaders?  How  will  the  geography  and  infrastructure 
impact  operations?  Do  some  initial  research  to  prepare  to  discuss  with  customer. 
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4.14.2.  Target  View  Guide  -  External  Issue  -  Iteration  1 

“Understand  what  the  actual  or  prospective  customer  needs  to  know  about  the  target  from  the 
perspective  of  a  customer  ”  (“ Developing  Intelligence  Issue  Statements  ”  ) 

ITERATION#:  1 

DATE:  21  May  09 

INITIAL  ISSUE:  The  CINC  is  concerned  about  this  insurgency  and  has  asked  the  J2 
to  assess  the  likeliness  that  the  insurgent  group  will  overthrow  General  Alpha. 

What  does  the  customer  want  to  know  about  the  target,  what  is  their  focus,  and  what  are  their 
priorities?  Why  is  the  issue  an  issue?  Why  is  it  important  to  address  this  issue? 

Knowns:  Country  Orange  is  in  a  region  of  strategic  importance  to  the  US.  Its 
actions  have  made  life  unpleasant  for  bordering  Country  Green,  which  is  friendly  to 
US  interests.  It  is  reported  that  Gen  Alpha  may  have  intentions  of  extending  his 
dominance  to  include  Country  Green  (reliability  unknown).  Country  Orange  has  a 
small  Air  Force.  Its  capabilities  are  well  known  by  us.  The  opposition  (Blues)  to 
Gen  Alpha  resides  in  Country  Green  and  has  asked  for  US  assistance  in  any 
insurgency.  There  are  implications  for  both  US  forces  and  U.S.  strategic  goals. 

Unknowns:  exactly  what  the  customer  wants  to  know,  their  focus  and  priorities. 
Seems  they  would  want  to  know  Gen  Alpha's  true  intentions  and  the  capabilities  of 
Country  Oranges  small  Air  Force  relative  to  what  the  US  plans  to  do.  We  can 
provide  Orange's  capabilities.  We  know  little  about  the  Blues'  capabilities,  and  they 
will  likely  need  to  know  that.  Do  initial  research  on  what  we  know  now  about  Gen 
Alpha's  and  Country  Green's  intentions;  also  research  Blues'  capabilities  by 
reviewing  production  and  talking  to  the  organization  responsible  for  assessing 
insurgent  capabilities.  Then  discuss  with  the  J2  what  is  important  to  them,  what 
they  know  and  their  sources.  Do  they  just  want  to  know  the  answers  or  do  they 
want  us  to  explain  all  the  conditions  associated  with  what  is  happening  and  potential 
success  or  failure  by  the  Blues. 

Discuss  with  customer:  Are  there  potentially  humanitarian  implications  (for  airlift, 
support,  etc)  if  Gen  Alpha  is  overthrown? 

Is  the  situation  of  interest  to  the  customer  an  emerging  condition?  Is  the  situation  recent, 
unexpected,  new? 

Known:  Situation  has  been  in  place  (dictatorship  in  unfriendly  country,  with 
ruthless  actions  on  populace  and  unpleasant  actions  towards  US  friendly  neighbor, 
Country  Green)  for  about  5  years.  The  new  information  from  X  and  Y  is  that  the 
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stage  was  set  for  an  insurgent  operation  against  Gen  Alpha  to  be  launched 
sometime  next  month. 

Unknowns:  Was  this  unexpected?  What  is  the  extent  of  Country  Green's 
involvement?  Discuss  with  the  customer. 

Is  the  issue  surrounding  the  target  a  “puzzle  or  a  mystery”?  Whether  the  customer  wants  to 
know  about  a  puzzle  or  mystery  is  important  to  understand  as  you  engage  with  the  customer. 
Discuss  the  information  added  below  with  the  customer. 

•  A  puzzle  is  knowable,  but  not  yet  by  us:  If  the  info  from  X  and  Y  is  reliable 
and/or  insurgents  have  decided  what  to  do,  then  it  is  a  puzzle, 
knowable,  but  not  yet  by  us.  Discuss  with  the  customer:  the 
importance  of  them  knowing  the  reliability  of  the  information;  do  initial 
research  to  understand  what  it  will  take  to  assess  this  reliability. 
Another  important  thing  for  the  customer  to  know,  which  is  unknown  is: 
Country  Green's  role... have  they  decided  what  to  do  or  not? 

•  A  mystery  can  be  known  by  no  one,  because  it  hasn’t  yet  been  decided:  There  is 
a  possibility  though  that  the  info  from  X  and  Y  is  not  reliable,  and 
perhaps  no  stage  has  been  set  for  an  insurgency.  That  is,  no  decision 
has  been  made  to  launch  the  insurgency  sometime  next  month.  The 
insurgency  leaders  may  have  not  yet  decided  what  to  do,  then  it  is  a 
mystery. 

Consider  the  following  to  think  about  what  you  know  and  what  is  unknown  about  the  customer’s 
primary  interests  in  the  target.  XXXCOM  is  a  military  organization  but,  given  their 
mission,  their  need  to  answer  this  question  may  have  diplomatic,  political,  economic, 
social  (humanitarian),  information,  etc  dimensions.  Plus  there  are  many  other 
things  on  the  list  that  they  will  be  interested  in  such  as  infrastructure.  Areas  that 
I  know  would  be  of  interest  to  the  customer  are  applicable  are  marked  below.  Talk 
to  customer  and  redo  when  answers  are  available  to  questions  on  the  customer  view 
guide  and  this  guide. 

•  Intentions,  Motivations  XX 

•  Military  XX 

•  Diplomatic,  political,  legal 

•  Economic 

•  Cultural,  religious,  sociological,  historical 

•  Information,  media,  propaganda 

•  Infrastructure,  telecommunications,  transportation 

•  Weather 
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•  Medical 

•  Criminal 

•  Governmental  structure,  plans,  policies 

•  Commercial,  Business 

•  Scientific/technical 

What  is  the  time  period  that  the  customer  wants  to  know  about?  E.g.,  historical,  current,  future 
(if  so,  how  far  in  the  future) 

Known:  Initial  question  deals  with  next  month.  But  that  will  raise  more  long-term 
questions  depending  on  what  is  assessed  about  next  month  and  associated 
capabilities,  intentions,  etc.,  plus  what  are  the  strategic  implications. 

Unknown:  time  period  the  customer  wants  addressed  in  the  analysis  we  give  them, 
(discuss  with  customer) 

Is  a  particular  geography  involved?  yes 

Known:  Country  Orange  includes  remote  rural  areas  and  two  urban  centers. 
Country  Green's  geography  is  known. 

Unknowns:  Blues'  primary  area  of  operations.  Seems  customer  would  want  to 
know,  discuss  with  them. 

Is  the  political  construct  of  interest  to  the  customer:  (Knowns  and  unknowns  are  marked) 

•  Traditional  -  yes,  Country  Orange  and  Country  Green 

•  Transnational  -  yes,  both  Orange  and  Green  involved 

•  Global  -  Unknown,  may  be  of  strategic  importance  to  others— does  the 
customer  want  to  know  this? 

•  Nation-state  -yes,  Orange  and  Green 

•  Non-state  actor  -  yes,  insurgency 

•  Single  actor  -  no 

•  Multiple  actors  -  yes,  insurgency,  Orange,  Green.  Unknown:  are  others  in 
the  region  involved— is  this  important  to  the  customer?  Discuss. 

Is  the  customer  interested  in  specific  regional  and  entity  dynamics? 

Known:  Relationship  between  Country  Orange  and  Country  Green  and  the 
insurgents  are  all  of  interest  and  important  for  the  customer  to  know. 

Unknown:  the  relationship  between  X  and  Gen  Alpha's  military.  Is  this  of  interest 
to  the  customer,  given  their  objective?  Does  the  customer  want  to  know  the 
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dynamics  of  other  players  in  the  region  and  beyond  the  region  for  those  with 
strategic  interests  (like  the  U.S.).  Discuss. 

Is  the  customer  interested  in  the  relationships  this  entity  might  have  with  other  countries  or  other 
entities?  See  above 

If  the  target  is  a  person,  what  is  of  interest  to  the  customer:  N/A 

•  What  are  the  goals  of  that  person? 

•  Is  the  study  to  be  a  biography? 

•  Is  the  objective  of  the  customer  to  predict  the  future  actions  of  that  person? 

•  Is  the  person  involved  with  a  particular  institution  or  organization? 

Is  a  specific  technology  part  of  what  the  customer  wants  to  know  about  the  issue?  Are  patents 
potentially  of  interest?  If  so,  what  aspects  of  the  technology  is  the  customer  interested  in.  N/ A 

If  the  target  is  a  system,  which  of  the  systems’  capabilities  are  of  interest  to  the  customer?  N/A 

If  the  target  is  a  process,  what  aspects  of  the  process  are  of  interest  to  the  customer?  N/A 

If  the  target  is  an  organization,  what  is  the  organizational  form?  Is  the  customer  interested  in 
the  nature  of  the  political  and  economic  system  that  the  organization  works  within?  N/A 

•  NGO 

•  Government  owned  or  controlled  business 

•  Private  business 
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4.14.3.  Customer  View  Guide  -  External  Issue  -  Iteration  2 


“Aspects  of  the  person  or  organization  who  is  the  intended  audience  for  the  intelligence  issue” 
“  Who  is  posing  the  intelligence  issue  and  how  are  they  planning  to  use  the  assessment?  ” 
(“Developing  Intelligence  Issue  Statements  ”) 

Changes  from  Iteration  1  are  highlighted. 

ITERATION#:  2 


DATE: 


23  May  09 


INITIAL  ISSUE:  The  CINC  is  concerned  about  this  insurgency  and  has  asked  the  J2 
to  assess  the  likeliness  that  the  insurgent  group  will  overthrow  General  Alpha. 

Who  is  the  customer  (organization,  person,  or  role)? 


Known:  XXXCO M/J2 


Unknowns:  None 


What  is  the  customer’s  organizational  affiliation?  What  is  the  reporting  hierarchy  for  the 
customer  (customer’s  organization  one  level  up? 

Known:  XXXCOM/J2  (part  of  XXX  Command) 


Known:  J2  is  the  intelligence  organization  that  works  for  the  CINC.  J2  reports 
directly  to  the  CINC. 

What  objective  is  the  customer  trying  to  achieve  by  position  this  issue?  What  is  the  problem  he 
or  she  is  trying  to  solve?  How  will  this  effort  help  the  customer  fulfill  his/her  mission?  How  is 
the  result  of  the  intelligence  effort  going  to  be  used? 


The  CINC  reports  directly  to  the  SECDEF. 


Knowns:  The  stated  objective  is  that  the  customer  has  been  asked  by  the  CINC  to 
"assess  the  likeliness  that  the  insurgent  group  will  overthrow  General  Alpha."  The 
J2's  role  is  to  provide  intelligence  to  support  the  CINC. 

The  underlying  objective  of  the  CINC  is  to  advise  the  SECDEF  on  potential  US 
assistance  to  insurgents;  and  if  the  decision  is  made  to  assist,  to  put  the  necessary 
plans  in  place.  The  consequences  of  the  insurgents  failing  are  something  the 
customer  wants  to  know. 

The  J2  wants  our  organization's  independent  assessment  of  the  likelihood 
overthrowing  General  Alpha.  The  J2  is  asking  us  and  one  other  intelligenc 
organization  (i.e.,  xxxxx)  to  do  an  independent  assessment.  We  have  talked  with 
that  organization.  The  J2  then  plans  to  make  their  own  independent  assessment 
and  provide  alternative  views  to  the  CINC.  The  J2  will  brief  the  CINC.  A 
representative  from  our  organization  will  be  invited  to  attend. 
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What  are  the  circumstances  that  led  to  the  request  for  this  intelligence  issue  to  be  addressed? 

Known:  Information  from  X  and  Y  (opposition  leaders)  to  CIA  operatives  that 
stage  was  set  for  an  insurgent  operation  against  Gen  Alpha  sometime  in  the  next 
month. 

See  policy  and  planning  circumstances  below. 

What  is  the  context  of  the  issue  in  terms  of  government  plans  and  policies? 


Knowns:  The  insurgency  wants  the  US.to  assist  them  with  military  and  other  aid. 
If  the  US  makes  a  decision  to  do  this,  US  mijitary  forces  must  prepare  to  assist 


the  insurgency.  In  addition,  plans  and  capabilities  must  be  in  place  for  non- 
combatant  evacuation  or  humanitarian 


US  policy  is  to  support  opposition  to  Gen  Alpha.  The  extent  of  that  support  will 
depend  on  exact  circumstances  and  timing.  Some  plans  are  in  place  and  must  be 
further  developed  based  on  the  support  required.  The  assessment  is  needed  to 
support  the  decision  being  made  and,  if  necessary,  further  develop  those  plans. 


Is  the  result  of  the  intelligence  effort  intended  to  support: 


• 

Short-term  or  long-term  effort:  Question  is  short-term,  consequence  of 

insurgency  failure  has  long-term  implications. 

• 

(Tactical  decision  making:  Not  1  u: 

• 

Weapons  system  acquisition:  No 

• 

^Policy  formulation: 

• 

Support  of  policy  execution: 

• 

Provocations  to  cause  other  side  to  act:  No 

• 

Operational: 

• 

(Planning: 

>  Long  range 

>  Strategic 

>  Operational 

>  Tactical 


What  is  the  customer’s  time  frame  for  the  product? 

Knowns:  There  is  information  indicating  insurgents  may  act  sometime  next  month 
(based  on  info  provided  to  CIA  operatives). 

Research  indicates  the  information  is  highly  reliable  from  a  well-vetted  source.  In 
addition,  there  is  corroborating  information  from  two  other  sources,  also  highly 

reliable. 
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The  customer  wants  the  product  in  7  calendar  days. 

What  is  the  customer’s  time  frame  for  his  or  her  decision? 

Known:  The  decision  is  expected  to  be  made  in  about  14  calendar  days. 

What  form  should  the  intelligence  assessment  take? 

Known:  It  is  to  be  a  concise  narrative  assessment,  with  graphics  to  support  text. 
An  assessment  is  to  be  made  of  the  likelihood  that  the  overthrow  will  occur  in  the 
next  month,  the  likelihood  of  success,  and  the  consequences  of  failure.  Prior  to 
the  information  being  provided  to  the  CINC,  we  will  have  interactive  discussions 
with  the  J2  staff  to  discuss  our  assessment. 

What  is  the  breadth  and  detail  desired  in  the  result? 

Knowns:  Outline  the  evidence  that  indicates  the  overthrow  will  occur  in  the  next 
month,  the  evidence  that  introduces  uncertainties  into  the  overthrow  occurring  in 
the  next  month,  conditions  for  success  and  failure  and  evidence  supporting  each, 
consequences  of  failure  and  rational  for  assessing  those  consequences.  Supporting 
information  should  include  the  insurgents'  capabilities  and  their  ability  to  execute 
surprise  attack,  including  the  likelihood  of  cooperation  of  other  elements  that  will 
contribute  to  success  or  failure. 
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4.14.4.  Target  View  Guide  -  External  Issue  -  Iteration  2 

“Understand  what  the  actual  or  prospective  customer  needs  to  know  about  the  target  from  the 
perspective  of  a  customer  ”  (“Developing  Intelligence  Issue  Statements  ”) 


ITERATION#:  2 


DATE: 


23  May  09 


Changes  from  Iteration  1  are  highlighted. 


INITIAL  ISSUE:  The  CINC  is  concerned  about  this  insurgency  and  has  asked  the  J2 
to  assess  the  likeliness  that  the  insurgent  group  will  overthrow  General  Alpha. 

What  does  the  customer  want  to  know  about  the  target,  what  is  their  focus,  and  what  are  their 
priorities?  Why  is  the  issue  an  issue?  Why  is  it  important  to  address  this  issue? 


Knowns:  Country  Orange  is  in  a  region  of  strategic  importance  to  the  U.S.  Its 
actions  have  made  life  unpleasant  for  bordering  Country  Green,  which  is  friendly  to 
US  interests.  It  is  reported  that  Gen  Alpha  may  have  intentions  of  extending  his 
dominance  to  include  Country  Green  (reliability  unknown).  Country  Orange  has  a 
small  Air  Force.  Its  capabilities  are  well  known  by  us.  The  opposition  (Blues)  to 
Gen  Alpha  resides  in  Country  Green  and  has  asked  for  US  assistance  in  any 
insurgency.  There  are  implications  for  both  US.forces  and  U.S.  strategic  goals. 

The  customer  wants  to  know  how  likely  it  is  that  the  overthrow  will  occur  in  the 
next  month,  the  conditions  for  success  and  failure,  the  insurgent's  capabilities, 
'ncluding  their  ability  to  execute  a  surprise  attack,  and  the  level  of  cooperation  (or 
ack  of)  that  can  be  expected  from  parties  internal  and  external  to  the  country 
that  could  contribute  to  success  or  failure  of  the  overthrow.  The  customer  will 
use  the  information  to  make  plans  to  assist  the  insurgency.  This  will  require  that 
we  know  Blues'  capabilities  and  the  capabilities  and  intentions  of  those  who  may 
either  assist  (e.g.,  potentially  Country  Green,  others)  or  impede  the  overthrow, 
ncluding  Gen  Alpha  and  his  Air  Force,  and  include  all  in  the  product. 


The  customer  is  also  planning  for  potential  non-combatant  evacuation  and/or 
humanitarian  support.  The  primary  assessment  needed  to  support  this  planning  wi 
be  done  by  another  organization,  but  we  should  share  any  information  that  may  be 
relevant  with  that  organization.  However,  this  will  not  be  focus  of  our  assessmenl 
and  the  resulting  product. 


Is  the  situation  of  interest  to  the  customer  an  emerging  condition?  Is  the  situation  recent, 
unexpected,  new? 


Known:  Situation  has  been  in  place  (dictatorship  in  unfriendly  country,  with 
ruthless  actions  on  populace  and  unpleasant  actions  towards  US  friendly  neighbor, 
Country  Green)  for  about  5  years.  The  new  information  from  X  and  Y  is  that  the 
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stage  was  set  for  an  insurgent  operation  against  Gen  Alpha  to  be  launched 
sometime  next  month. 


The  customer  expected  that  an  insurgent  operation  would  be  launched  within  the 
next  year  and  some  planning  had  been  done.  However,  the  recent  information 
accelerates  the  customer's  need  for  the  assessment  to  support  rapid  planning. 


Is  the  issue  surrounding  the  target  a  “puzzle  or  a  mystery”?  Whether  the  customer  wants  to 
know  about  a  puzzle  or  mystery  is  important  to  understand  as  you  engage  with  the  customer. 


A  puzzle  is  knowable,  but  not  yet  by  us:  The  info  from  X  and  Y  has  b :: : 


n 

ided 


determined  to  be  reliable,  indicating  that  the  insurgents  have  decide 
what  to  do.  Consequently,  it’s  a  puzzle.  However,  we  must  be  vigilant 
For  any  indications  of  contradictory  information  and  changes  in  Blues 
ulans  due  to  changing  circumstances 


A  mystery  can  be  known  by  no  one,  because  it  hasn’t  yet  been  decided: 


that  the  customer  doesn't  know  Country's  Green's  expected  role  but 
wants  to  know.  So  we  must  include  understanding  it,  and  the 


mplications,  in  our  analysis.  It  is  not  known  if  Country  Green  has  made  a 
decision  and  if  so  what  it  is.  So  we  don't  if  that's  a  puzzle  or  mystery. 


Consider  the  following  to  think  about  what  you  know  and  what  is  unknown  about  the  customer’s 
primary  interests  in  the  target.  XXXCOM  is  a  military  organization  but,  given  their 
mission,  their  need  to  answer  this  question  may  have  diplomatic,  political,  economic, 
social  (humanitarian),  information,  etc  dimensions.  Plus  there  are  many  other 
things  on  the  list  that  they  will  be  interested  in  such  as  infrastructure.  Areas  that 
I  know  would  be  of  interest  to  the  customer  are  applicable  are  marked  below. 


•  Intentions,  Motivations  XX 

•  Military  XX 

•  Diplomatic,  political,  legal 

•  Economic 

•  Cultural,  religious,  sociological,  historical 

•  Information,  media,  propaganda 

•  Infrastructure,  telecommunications,  transportation 

•  Weather 

•  Medical 

•  Criminal 

•  Governmental  structure,  plans,  policies 

•  Commercial,  Business 

•  Scientific/technical 
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What  is  the  time  period  that  the  customer  wants  to  know  about?  E.g.,  historical,  current,  future 
(if  so,  how  far  in  the  future) 


Known:  The  customer  wants  to  know  what  will  happen  in  the  next  month  regarding 
the  overthrow.  They  also  want  to  know  the  consequences  of  failure,  specifically 
focusing  on  those  consequences  during  the  next  year. 


Is  a  particular  geography  involved?  yes 

Known:  Country  Orange  includes  remote  rural  areas  and  two  urban  centers. 
Country  Green's  geography  is  known.  The  customer  wants  details  on  Blues'  primary 
areas  of  operations  to  support  the  customer's  planning. 


Is  the  political  construct  of  interest  to  the  customer:  (Knowns  and  unknowns  are  marked) 


•  Traditional  -  yes,  Country  Orange  and  Country  Green 

•  Transnational  -  yes,  both  Orange  and  Green  involved 


• 

Global  - 

Not  a  priority  of  the  customer.  Will  only  come  into  play  if 

there  are  consequences  of  the  overthrow  failing  that  will  manifest 

themselves  in  about  the  next  year. 

•  Nation-state -yes,  Orange  and  Green 

•  Non-state  actor  -  yes,  insurgency 

•  Single  actor  -  no 


•  Multiple  actors  -  yes,  insurgency,  Orange,  Green,  The  customer  wants  to 
know  the  roles  of  others  that  may  assist  or  impede  the  overthrow. 

Is  the  customer  interested  in  specific  regional  and  entity  dynamics? 

Known:  Relationship  between  Country  Orange  and  Country  Green  and  the 
insurgents  are  all  of  interest  and  important  for  the  customer  to  know. 

The  relationship  between  X  and  Gen  Alpha's  military  is  of  interest  to  the  customer 
because  it  may  have  implications  regarding  Gen  Alpha's  ability  to  impede  the 
overthrow.  The  customer  does  want  to  know  the  dynamics  of  other  players  in  the 
region,  as  well  as  those  beyond  the  region  with  strategic  interests,  if  those 
dynamics  would  generate  consequences  in  the  next  year  under  the  condition  that 
the  overthrow  failed. 

Is  the  customer  interested  in  the  relationships  this  entity  might  have  with  other  countries  or  other 
entities?  See  above 


If  the  target  is  a  person,  what  is  of  interest  to  the  customer:  N/A 

•  What  are  the  goals  of  that  person? 

•  Is  the  study  to  be  a  biography? 
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•  Is  the  objective  of  the  customer  to  predict  the  future  actions  of  that  person? 

•  Is  the  person  involved  with  a  particular  institution  or  organization? 

Is  a  specific  technology  part  of  what  the  customer  wants  to  know  about  the  issue?  Are  patents 
potentially  of  interest?  If  so,  what  aspects  of  the  technology  is  the  customer  interested  in.  N/ A 

If  the  target  is  a  system,  which  of  the  systems’  capabilities  are  of  interest  to  the  customer?  N/A 

If  the  target  is  a  process,  what  aspects  of  the  process  are  of  interest  to  the  customer?  N/A 

If  the  target  is  an  organization,  what  is  the  organizational  form?  Is  the  customer  interested  in 
the  nature  of  the  political  and  economic  system  that  the  organization  works  within?  N/A 

•  NGO 

•  Government  owned  or  controlled  business 

•  Private  business 
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4.14.5.  Intelligence  Issue  Statement  -  External  Issue 

“Framing  gives  a  broad  form  to  an  issue  and  states  the  goals  and  objectives  of  the 
analysis  ...deliberately  limits  the  scope  of  the  inquiry  ...tells  you  where  you  will  not  go  while 

trying  to  address  the  intelligence  issue.  ” 

“provides  direction  for  the  intelligence  effort  ...reflects  your  understanding  of  what  you  intend  to 
explore  in  the  intelligence  effort  and  why  the  effort  is  being  undertaken  ...synthesizes  the 
knowledge  you  gained  while  framing  the  issue  from  the  customer  and  target  views.  ” 

INTELLIGENCE  ISSUE  STATEMENT  (24  May  09):  Understand  the  likelihood  that  the 
Blues  will  overthrow  Gen  Alpha  in  the  next  month  to  support  advice,  decision 
making,  and  planning  relative  to  providing  U.S.  assistance  to  the  Blues.  Understand 
the  conditions  for  success  and  failure  (with  evidence  supporting  each)  and  the 
consequences  of  failure  (including  the  rationale  for  assessing  those  consequences), 
focusing  on  the  consequences  during  the  next  year.  The  product  is  due  NLT  31 
May  09. 

INITIAL  ISSUE  FROM  CUSTOMER:  The  CINC  is  concerned  about  this  insurgency 
and  has  asked  the  J2  to  assess  the  likeliness  that  the  insurgent  group  will 
overthrow  General  Alpha. 

GOALS  AND  OBJECTIVES 

•  Customer(s)  requesting  the  effort,  by  name,  organization,  or  function;  or  the  same 
information  describing  prospective  customers:  XXXCO M/  J2 

•  Current  or  future  events  driving  the  intelligence  issue:  Country  Orange  is  in  a 
region  of  strategic  importance  to  the  U.S.  Its  leader,  Gen  Alpha,  is  a 
dictator  unfriendly  the  U.S.  Highly  reliable  information  indicates  that 
an  insurgent  group,  the  Blues,  plans  to  overthrow  Gen  Alpha  sometime 
during  the  next  month.  The  Blues  have  requested  U.S.  assistance,  and 
it  is  U.S.  policy  to  support  opposition  to  Gen  Alpha. 

•  Target  of  the  effort:  The  Blues  and  their  likelihood  of  success  or  failure  in 
conjunction  with  those  players  and  conditions  that  may  help  them 
succeed  or  impede  their  efforts. 

•  What  the  customer  intends  to  do  with  the  results  of  the  intelligence  effort  or  how  a 
prospective  customer  might  use  the  analysis  to  support  their  mission:  The 

assessment  will  be  given  to  the  CINC  to  support  decision  making  on 
whether  to  support  the  Blues,  the  assistance  provided,  and  developing 
plans  to  execute  those  decisions. 

FOCUS  (What  the  customer  needs  to  know  about  the  tar  get...  scope  of  the  intelligence 
effort. ..and  coverage  of  the  target):  The  customer  wants  to  know  how  likely  it  is  that 
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the  overthrow  will  occur  in  the  next  month,  the  conditions  for  success  and  failure, 
the  insurgents'  capabilities,  including  their  ability  to  execute  a  surprise  attack,  and 
the  level  of  cooperation  (or  lack  of)  that  can  be  expected  from  parties  internal  and 
external  to  the  country  that  could  contribute  to  success  or  failure  of  the 
overthrow.  The  customer  will  use  the  information  to  make  plans  to  assist  the 
insurgency.  This  will  require  that  we  know  Blues'  capabilities  and  the  capabilities 
and  intentions  of  those  who  may  either  assist  (e.g.,  potentially  Country  Green, 
others)  or  impede  the  overthrow,  including  Gen  Alpha  and  his  Air  Force,  and  include 
all  in  the  product. 

SPECIAL  CONSIDERATIONS  (  ...notes,  guidelines,  or  other  factors  that  might  be  related  to 
the  intelligence  issue  statement.  This  section  may  also  be  used  to  elaborate  those  issues  that  will 
not  be  considered  in  this  intelligence  effort.):  The  customer  is  also  planning  for  potential 
non-combatant  evacuation  and/or  humanitarian  support.  The  primary  assessment 
needed  to  support  this  planning  will  be  done  by  another  organization,  but  we  should 
share  any  information  that  may  be  relevant  with  that  organization.  However,  this 
will  not  be  focus  of  our  assessment  and  the  resulting  product. 

BIBLIOGRAPHY  (...existing  intelligence  products  that  are  related  to  the  issue...  that  seem  to 
address  some  or  all  of  the  intelligence  issue  under  consideration):  None  answer  the  issue. 
There  are  some  supporting  products  that  deal  with  aspects  of  Country  Orange's 
Air  Force,  the  Blues,  and  Country  Green. 

•  Products  that  address  (in  part)  relevant  aspects: 

>  _ 

>  _ 

>  _ 

>  _ 

>  _ 
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4.14.6.  The  Case  of  General  Alpha:  Facing  Insurgency? 

DEFENSE  ATTACHE’S  BACKGROUND  REPORT  ON  COUNTRY  ORANGE 

In  a  certain  part  of  the  world  of  strategic  importance  to  the  U.S.,  a  country  named  Orange 
[population  around  2.5  million]  has,  for  five  years,  been  in  the  throes  of  a  dictatorship  that  has 
decidedly  unfriendly  inclinations  toward  the  U.S.  We  presently  have  no  diplomatic  relations 
with  Orange.  In  addition,  this  dictatorial  government  has  adopted  quite  ruthless  methods  to 
maintain  its  hold  on  the  populace  of  Orange.  Further,  the  ruler  in  Orange,  General  Alpha,  has 
made  life  unpleasant  for  a  bordering  country,  Green,  friendly  to  U.S.  interests,  by  fomenting 
repeated  border  disputes.  It  seems  clear  that  Gen.  Alpha  has  every  intention  of  extending  his 
dominance  to  include  this  bordering  country  Green.  Gen.  Alpha's  support  has  come  mainly  from 
the  military  forces  in  Orange  and  from  individuals  in  the  two  major  urban  areas  in  Orange  who 
have  profited  by  cooperating  with  Gen.  Alpha.  The  majority  of  Orange's  population  lives  in 
rural  areas,  many  of  which  are  quite  remote  from  the  two  Orange  urban  centers,  cities  A  and  B. 
Opposition  to  Gen.  Alpha  among  the  general  urban  and  rural  population  in  Orange  has  been  held 
in  check  due  to  very  ruthless  measures  taken  by  Gen.  Alpha's  military  against  persons  Alpha  has 
called  "undesirables."  Many  of  these  "undesirables"  have  either  been  publicly  executed  or  now 
languish  in  military  prisons.  The  military  in  Orange,  numbering  about  75,000,  is  controlled 
exclusively  by  Gen.  Alpha  and  seems  well  equipped  for  ground  fighting.  In  addition,  Gen. 

Alpha  has  a  small  air  force  consisting  mainly  of  propeller-driven  aircraft  and  a  few  helicopters. 
Infonnation  about  events  in  Orange  is  generally  hard  to  obtain.  International  news  organizations 
such  as  CNN  have  been  barred  from  entering  Orange  since  Gen.  Alpha  wrested  control  of 
Orange  about  five  years  ago.  Gen.  Alpha’s  government  controls  the  press,  radio,  and  the  single 
television  station  that  is  allowed  to  broadcast. 

Quite  recently,  an  opposition  group,  named  the  Blues,  has  come  into  existence  and  is  led  by  two 
persons  who  have  managed  to  avoid  capture  by  Gen.  Alpha's  military.  Person  X  was  a  high- 
ranking  military  leader  in  country  Orange  before  its  government  was  overthrown  by  Gen.  Alpha. 
Person  Y  is  a  former  professor  of  political  science  at  the  only  university  in  Orange  and  who  also 
held  a  high-ranking  political  position  in  Orange  before  the  government  was  overthrown.  Both  X 
and  Y  have  remained  in  hiding  in  a  remote  region  of  Orange  and  have,  from  time  to  time,  been 
granted  sanctuary  in  neighboring  country  Green.  When  they  were  last  in  Green,  about  a  week 
ago,  X  and  Y  told  our  CIA  operatives  that  the  stage  was  set  for  an  insurgent  operation  against 
Gen.  Alpha  and  his  ruthless  regime  in  country  Orange.  Furthermore,  they  indicated  that  their 
insurgency  operation  would  be  launched  sometime  in  the  next  month.  Of  course  they  asked  for 
military  and  other  assistance  from  the  U.S. 

TASKING 

•  You  are  a  member  of  a  team  of  military  capabilities  analysts. 

•  The  CINC  is  concerned  about  this  insurgency  and  has  asked  the  J2  to  assess  the 
likeliness  that  the  insurgent  group  will  overthrow  General  Alpha.74 

Customer  Organization:  XXXGOM/J275 


74  Hughes,  F.J.  &  Schum,  D.A.  The  Art  and  Science  of  the  Process  of Intelligence  Analysis.  Case  Study  #1  (the  Case 
of  General  Alpha:  Facing  Insurgency?).  Analysis.  2006. 

75  Added  to  the  Hughes/Schum  Case  Study  #1  for  the  purposes  of  this  example. 
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5.0  SETTING  A  DIRECTION:  THE  ANALYSIS  PLAN 

5.1  Introduction 

Doing  effective  intelligence  analysis  requires  knowledge  of  the  target’s  capabilities,  motivations, 
intentions,  culture,  past  behavior,  threat  perceptions,  relationships,  and  a  myriad  of  other  factors. 
You  have  probably  acquired  some  of  this  knowledge  while  you  were  creating  the  intelligence 
issue  statement.  How  can  you  weave  this  knowledge  to  address  the  intelligence  issue  covered  by 
your  AOR  (A  OR  issue),  self-initiated  by  you  ( self-initiated  issue),  or  posed  by  a  customer 
(< external  issue)?  How  should  you  proceed  once  you  feel  you  understand  the  intelligence  issue? 
How  can  you,  as  an  intelligence  analyst,  identify  the  most  fruitful  direction  for  further 
investigation  and  use  your  time  wisely?  We  refer  to  these  considerations  about  the  subject 
matter  associated  with  the  intelligence  issue  as  the  substantive  areas  of  analysis.  The  substantive 
areas  of  analysis  are  what  you  need  to  know  to  address  the  intelligence  issue. 

This  FAC  section  will  provide  approaches  for  thinking  about  and  identifying  the  substantive 
areas  of  analysis.  In  addition,  this  FAC  section  will  also  provide  approaches  for  some  practical 
issues  you’ll  need  to  address  to  accomplish  your  analysis.  For  example,  what  resources  will  you 
use  to  address  the  intelligence  issue?  What  is  the  schedule  you  will  need  to  follow  in  order  to 
meet  your  own  deadlines  or  deadlines  set  by  the  customer?  Planning  the  activities  involved  in 
addressing  an  intelligence  issue,  whether  an  AOR  issue,  a  self-initiated  issue,  or  an  external 
issue,  is  an  essential  and  integral  part  of  the  intelligence  analysis  process.  In  other  areas  of 
endeavor,  this  planning  is  often  referred  to  as  a  project  management  plan.  However,  with 
intelligence  analysis  there  are  additional  complexities  and  important  differences  that  are  not 
covered  by  traditional  project  planning  plans.  These  will  also  be  discussed  in  this  section. 

So,  in  summary,  at  this  stage  of  the  FAC  you  apply  both  knowledge  about  your  area  of  analytic 
responsibility  and  planning  techniques  in  order  to  address  the  intelligence  issue.  How  do  we 
integrate  these  two  demands  on  the  intelligence  analyst? 

We  recommend  that  the  infonnation  about  the  substantive  areas  of  your  analysis  effort,  as  well 
as  resources,  schedules,  and  other  activities,  be  incorporated  into  a  written  document  we  call  the 
analysis  plan.  The  analysis  plan  describes  everything  needed  to  address  an  intelligence  issue:  all 
the  areas  of  intelligence  analysis  that  are  necessary,  what  will  be  done,  what  resources  are 
required,  who  will  do  the  work,  and  when  the  work  needs  to  be  completed.  It  describes  how  the 
analysis  effort  will  be  structured  to  facilitate  the  acquisition  and  analysis  of  the  infonnation 
needed.  The  analysis  plan  aids  coordination  and  communication  and  provides  a  basis  for  you  to 
monitor  your  progress  on  the  intelligence  analysis  effort.  In  addition,  the  act  of  creating  an 
analysis  plan  often  forces  you  to  consider  aspects  of  the  effort  that  would  escape  your  attention  if 
you  were  not  engaged  in  planning. 

In  this  section  of  the  FAC  we  propose  an  approach  to  help  you  identify  and  elaborate  the 
substantive  areas  involved  in  your  analysis  effort.  These  substantive  areas  are  the  essential 
starting  point  for  planning  and  doing  your  analysis.  We  then  provide  some  basic  infonnation 
about  creating  an  analysis  plan. 
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Where  does  the  creation  of  the  analysis  plan  Jit  into  the  analysis  process? 

In  the  “Introduction”  section  of  the  FAC  we  discussed  the  intelligence  analysis  process  first 
described  by  Sherman  Kent.  Kent’s  process  included  the  following  steps: 

1 .  The  appearance  of  a  problem  requiring  the  attention  of  a  strategic  intelligence  staff. 

2.  Analysis  of  this  problem  to  discover  which  facets  of  it  are  of  actual  importance  to  the 
U.S.  and  which  of  several  lines  of  approach  are  most  likely  to  be  useful  to  its 
governmental  customers. 

3.  Collection  of  data  bearing  upon  the  problem  as  formulated  in  stage  2.  This  involves  a 
survey  of  data  already  at  hand  and  available  in  the  libraries  of  documentary  materials, 
and  an  endeavor  to  procure  new  data  to  fill  in  gaps. 

As  Kent  described  in  step  2,  when  you  developed  your  intelligence  issue  statement,  you  began 
your  analysis  of  the  intelligence  problem.  You  identified  those  facets  of  the  issue  that  are  of 
importance  to  your  actual  or  prospective  customers.  Your  analysis  of  the  intelligence  issue  will 
continue  as  you  produce  the  analysis  plan.  By  further  exploring  and  elaborating  the  substantive 
knowledge  needed  to  address  your  intelligence  issue  statement,  you  lay  the  groundwork  for  step 
3,  which  we  refer  to  as  information  acquisition  in  the  FAC. 

What  are  the  benefits  of  setting  a  direction  for  an  intelligence  analysis  effort? 

It  is  important  to  set  the  direction  of  an  intelligence  analysis  effort  because: 

•  (i.e.,  what  you  need  to  know  to  address  the  intelligence  issue)  need  to  be  identified 
early  in  the  analysis  process.  All  other  activities  in  the  intelligence  analysis  effort 
flow  from  this  identification,  so  it  is  essential  that  it  occur  early  in  the  analysis 
process. 

•  Knowledge  of  the  substantive  areas  of  analysis  is  especially  important  for  the 
acquisition  of  information  necessary  to  address  to  the  intelligence  issue. 
Identification  of  what  you  need  to  know  will  guide  your  infonnation  planning  and 
subsequent  information  acquisition. 

•  Intelligence  analysis  often  occurs  under  critical  time  constraints.  Making  the  most 
of  the  resources  and  time  available  to  you  is  essential. 

•  Plans  permit  you  to  monitor  and  adjust  your  progress  during  the  intelligence 
analysis  effort. 

•  Creating  an  analysis  plan  doesn’t  guarantee  that  your  intelligence  analysis  effort 
will  be  problem- free,  but  planning  does  decrease  the  risk  of  certain  kinds  of 
problems. 

•  An  understanding  of  the  effort’s  direction  provides  the  basis  for  guidance  to  others 
who  may  be  involved  in  planning  their  contributions  to  the  effort.  Understanding 
your  direction  will  aid  you  in  effective  collaboration  with  others  during  your  effort. 


76  Kent,  S.  Strategic  Intelligence  for  American  World  Policy,  Princeton  University  Press,  Princeton,  NJ,  1949,  pp. 
157-158. 
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•  A  description  of  the  activities  and  the  interdependencies  involved  in  an  intelligence 
analysis  effort  promotes  communication  between  all  of  the  parties  (e.g.,  infonnation 
providers,  other  analysts,  etc)  contributing  to  the  effort. 

Is  planning  an  intelligence  analysis  effort  different  than  planning  for  other  types  of  efforts? 

Some  aspects  of  an  intelligence  analysis  effort  can  be  managed  using  a  traditional  project 
management  approach.  Knowledge  of  basic  project  management  techniques  can  help  you  as  you 
plan  for  intelligence  analysis.  Some  aspects  of  an  intelligence  analysis  effort,  however,  are  very 
different  than  a  traditional  project.  The  crucial  difference  between  an  intelligence  analysis  effort 
and  other  projects  is  that  project  managers  for  traditional  projects  work  diligently  to  make  the 
project’s  definition  and  goals  measurable,  tangible,  and  verifiable  before  beginning.  For  reasons 
that  we  will  explain  later  in  this  section,  an  intelligence  analysis  effort  does  not  lend  itself  well  to 
a  definition  that  concrete.  An  intelligence  analysis  effort  can  contain  so  many  unknowns  that 
planning  is  more  difficult  than  a  traditional  project.  Our  objective  in  this  section  is  to  help  you 
plan  effectively  even  if  many  aspects  of  the  effort  seem  intangible. 

Who  is  the  audience  for  the  analysis  plan? 

The  analysis  plan  is  created  by  you,  for  you,  and  describes  the  substantive  areas  and  plans  for 
your  analysis  activities.  It  is  another  instance  of  the  FAC’s  “writing  as  thinking”  emphasis. 
Identifying  the  knowledge  you  need  in  order  to  carry  out  an  effective  intelligence  analysis 
effort  is  a  major  cognitive  challenge.  As  we  discussed  earlier,  creating  a  written  representation  is 
an  important  part  of  helping  you  think  more  effectively  about  the  intelligence  issue  and  your 
analysis  efforts.  The  analysis  plan  is  the  written  representation  for  this  phase  of  the  FAC. 

Before  we  begin  discussing  the  specifics  of  creating  an  analysis  plan,  we  need  to  discuss  the 
preparatory  work  that  will  help  you  plan  successfully. 

5.2  Getting  from  Here  to  There 

A  project  is  a  temporary  endeavor  undertaken  to  create  a  unique  product,  sendee,  or  result.11 

Analysis  in  your  area  of  responsibility  is  ongoing.  As  long  as  you  are  assigned  to  an  area,  you 
accumulate  knowledge  and  information,  so  analysis  doesn’t  have  a  set  beginning  and  end.  When 
faced  with  a  specific  intelligence  issue  to  analyze,  you’ll  certainly  use  your  accumulated 
knowledge  and  assembled  infonnation  to  address  that  particular  intelligence  issue.  In  the  FAC, 
however,  the  analysis  effort  for  addressing  this  particular  issue  has  a  beginning  (development  of 
the  intelligence  issue  statement)  and  end  (a  product).  If  it  is  an  AOR  issue,  the  result  will  be 
achieving  understanding  of  your  AOR  and  there  will  be  many  resulting  products  for  actual  or 
prospective  customers.  The  analysis  effort  relies  on  a  group  of  resources  assembled  to 
accomplish  an  outcome  relating  to  the  intelligence  issue.  Considering  only  these  general 
characteristics,  an  intelligence  analysis  effort  is  similar  to  a  wide  variety  of  other  projects  such  as 
constructing  a  building,  creating  a  new  drug,  or  developing  a  software  application. 

However,  an  intelligence  analysis  effort  differs  from  other  classic  projects  in  several  important 
respects.  A  construction  manager  who  formulates  plans  for  an  office  building  knows,  usually 
from  the  beginning  of  the  project,  what  the  outcome  of  the  project  will  be.  That  is,  the  result  will 
be  an  office  building  that  looks  and  functions  quite  similarly  to  the  plans  developed  at  the 

77  Anonymous.  A  Guide  to  the  Project  Management  Body  of  Knowledge,  3rd  edition:  PMBOK©  Guide.  Project 
Management  Institute  Inc.,  Newtown  Square,  PA,  2004,  pp.  5. 

82 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 

88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


beginning  of  the  project.  In  addition,  the  approaches  to  developing  a  building  are  well-known 
and  commonly  practiced.  Finally,  it  is  relatively  easy  for  the  construction  manager  to  identify 
who  will  participate  in  the  project.  The  typical  construction  project  relies  on  a  well-understood 
path  to  a  known  outcome  with  resources  that  are  easily  identified. 

An  intelligence  analyst,  however,  knows  that  an  intelligence  issue  needs  to  be  addressed,  but  the 
exact  nature  of  the  outcome  of  addressing  the  issue  won’t  be  known  until  the  analysis  is 
complete.  In  the  case  of  an  AOR  issue,  there  will  be  many  analyses  and  resulting  products, 
typically  over  a  long-period  of  time.  Over  this  period  of  time,  there  could  be  new  emerging 
situations  in  the  AOR  that  will  change  your  understanding  and  plans  for  analysis.  An  intelligence 
analysis  effort  relies  on  many  components  and  participants  that  have  to  be  brought  together 
efficiently  in  order  to  address  the  issue.  In  addition,  the  activities  you’ll  engage  in  to  create  the 
assessment  (or  many  assessments  for  AOR  issues)  may  not  be  as  predictable  as  the  approaches 
used  for  construction  or  software  development,  for  example.  An  intelligence  analysis  effort  may 
also  require  resources  that  are  not  easily  identified  during  the  early  stages  of  the  effort  but 
become  necessary  based  on  the  intennediate  results  of  analysis. 

Creating  a  plan  for  an  intelligence  analysis  effort  can  seem  daunting  because  of  the  uncertainty 
surrounding  the  approach,  outcome,  and  the  participants  in  the  effort.  Some  existing  project 
management  techniques  could  be  applicable  to  intelligence  analysis,  but  the  fact  that  project 
management  assumes  known  outcomes  and  methods  seems  to  limit  its  usefulness.  How  can  you 
decide  what  resources  to  use  and  when  to  use  them  if  you  don’t  know  exactly  what  the  end  result 
of  your  planning  is? 

This  section  of  the  FAC  provides  an  approach  you  can  use  to  conquer  some  of  the  differences 
between  traditional  planning  and  planning  for  intelligence  analysis.  We  propose  a  method  that 
you  can  use  to  aid  you  in  identifying  and  understanding  of  the  substantive  areas  of  analysis  that 
you  will  need  to  address  for  the  effort.  By  exploring  what  you  need  to  know  to  address  the 
intelligence  issue,  including  how  potential  events  associated  with  your  target  may  unfold,  you 
can  identify  the  substantive  areas  associated  with  your  effort  more  specifically.  Once  these 
substantive  areas  are  identified,  you  can  assess  how  to  approach  each  substantive  area,  including 
who  will  need  to  contribute  to  your  efforts.  As  your  analysis  effort  continues  you  will,  of  course, 
augment  and  modify  these  substantive  areas  and  adapt  your  plans  and  activities  as  needed. 

5.3  Building  the  Analysis  Plan 

In  the  field  of  strategic  research  and  intelligence,  no  analyst  can  afford  to  assume  that  he  or  she 
already  knows  everything  there  is  to  know.  Nor  are  assessment  projects  likely  to  be  able  to  be 
completed  without  some  considerable  investment  in  time  and  other  resources.  In  this  arena, 
there  is  simply  too  much  effort  involved  to  risk  taking  anything  but  the  most  careful  approach 
possible.  This  means  committing  vourself  not  onlv  to  careful  implementation  and  analvsis 

78 

throughout  the  project,  but  also  to  careful  and  comprehensive  planning  at  the  outset. 

Developing  an  analysis  plan  consists  of  the  following  activities: 


78McDowell,  D.  Strategic  Intelligence:  a  Handbook  for  Practitioners,  Managers  and  Users.  Istana  Enterprises  Inc., 
Pambula  NSW  Australia,  1998,  pp.  91. 
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•  Identify  the  substantive  areas  of  analysis.  You  may  find  it  helpful  to  identify  which 
areas  are  already  known  and  which  areas  are  unknowns  and  how  familiar  and 
current  you  are  with  those  areas  you  know  something  about. 

•  List  the  major  milestones  or  key  events  for  the  analysis  effort 

•  List  the  activities  that  must  be  completed,  by  you  and  others,  in  order  to  address  the 
intelligence  issue 

•  Identify  any  other  resources  needed  (e.g.  contractual  dollars),  if  applicable 

•  Develop  an  overall  schedule  for  the  effort,  based  on  your  needs  or  the  needs  of  the 
customer 

•  Write  the  analysis  plan 

I  think  I’m  going  to  need  an  example  to  understand  this... 

To  make  these  concepts  easier  to  understand,  we’re  going  to  use  an  example  to  discuss  how  to 
create  an  analysis  plan. 

5.3.1.  Identify  the  Substantive  Areas  of  Analysis 

In  order  to  illustrate  how  you  develop  the  substantive  areas  of  intelligence  analysis  for  an  issue 
we  use  “The  Case  of  General  Alpha:  Facing  Insurgency”  as  presented  in  subsection  3.14  of  the 
“Developing  Intelligence  Issue  Statements”  section  of  the  FAC.  It  is  easier  to  follow  the  example 
for  section  4  if  you  keep  the  samples  from  section  3  handy  -  you  will  need  to  refer  to  the 
intelligence  issue  statement,  customer  view,  and  target  view  guides  developed  for  the  example  as 
we  discuss  the  analysis  plan. 

NOTE 

We  have  designed  the  example  so  that  it  has  enough  detail  that  you  can  understand  the 
steps  involved  in  creating  the  analysis  plan,  but  we  did  limit  some  aspects  of  it  so  that  it 
would  work  well  as  a  concise  example. 


The  intelligence  issue  statement  created  for  the  case  study  is: 

Understand  the  likelihood  that  the  Blues  will  overthrow  General  Alpha  in  the  next  month 
to  support  advice,  decision  making,  and  planning  relative  to  providing  U.S.  assistance  to 
the  Blues.  Understand  the  conditions  for  success  and  failure  (with  evidence  supporting 
each)  and  the  consequences  of  failure  (including  the  rationale  for  assessing  those 
consequences),  focusing  on  the  consequences  during  the  next  year. 

In  order  to  create  the  analysis  plan  you  need  to  consider  a  wide  range  of  possibilities  for  what 
you  need  to  know  to  address  the  intelligence  issue.  Considering  how  future  events  may  unfold 
can  help  you  identify  what  you  need  to  know.  It  will  help  you  determine  the  substantive  areas  of 
analysis  you  will  need  to  pursue  in  order  to  address  this  issue. 

How  do  I  begin? 

Most  intelligence  issues  relate  to  the  status  of  developments  in  the  future  that  are  caused  by  a 
series  of  interrelated  events  stemming  from  human  actions.  “Will  an  earthquake  occur?”  is  not  an 
intelligence  issue  but  “Will  Country  X  invade  Country  Y?”  is.  A  simple  yes  or  no  assessment  is 
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inadequate  since  you  must  establish  the  status  of  the  developments  as  they  exist.  Thus  your 
analysis  effort,  and  the  analysis  plan  developed  to  accomplish  that  analysis,  must  be  target- 
oriented,  not  focused  on  what  your  final  assessment  will  be.  At  this  point,  you  want  to  avoid 
trying  to  think  about  the  end  result  of  your  analysis.  The  fact  that  you  don’t  know  the  end  result 
of  the  analysis  is  not  an  impediment  to  doing  the  plan. 

The  future  status  of  the  intelligence  target,  regardless  of  what  it  is,  is  the  result  of  an 
accumulation  of  related  events,  decisions,  and  accomplishments  by  those  associated  with  the 
target.  You  begin  your  work  on  the  analysis  plan  by  thinking  about  and  identifying  the  possible 
sequences  of  these  interactions  and  what  you  need  to  know  to  understand  them.  By  exploring 
how  potential  events  associated  with  your  target  may  unfold,  you  can  identify  more  specifically 
the  substantive  areas  that  you  will  need  to  address,  including  the  areas  where  you  may  need 
analytic  contributions  from  others.  Exploring  potential  events  is  something  you  do  solely  for  the 
purpose  of  planning  and  guiding  your  subsequent  analysis.  This  exploration  helps  you  initially 
scope  your  effort.  As  you  acquire  and  evaluate  information,  you  will  no  doubt  find  other  areas  to 
explore  and  other  potential  events  to  examine,  ah  of  which  will  shape  your  thinking  regarding 
the  intelligence  issue. 

In  order  to  help  your  thinking  about  the  possible  sequences  of  interactions  associated  with  the 
intelligence  issue,  consider  the  following  factors: 

•  Dynamics  and  interactions:  Includes  the  interaction  of  the  actors,  artifacts,  and 
environment  over  a  given  time  frame ;  what  decisions  would  have  to  be  made,  by 
whom,  and  how  those  decisions  might  be  made;  how  the  situation  might  evolve; 
and  what  might  take  place. 

>  Actors  are  the  person  or  persons  with  goals  and  objectives  related  to  the 
intelligence  issue.  Actors  may  also  be  organizations  of  ah  types,  formal  and 
infonnal,  including  governmental  and  non-governmental  organizations. 

>  Artifacts  are  the  inanimate  objects  central  to  the  intelligence  issue.  These 
could  include  equipment,  facilities,  locations,  and  nomenclature  associated 
with  these  artifacts. 

>  Environment  is  the  setting  for  the  action  or  the  situation  being  considered. 

>  Time  frame  is  the  duration  or  period  of  time  relevant  to  the  intelligence 
issue.  It  can  include  both  current  and  past  events,  as  well  as  the  future  time 
frame  you  are  considering. 

How  would  an  intelligence  analyst  use  these  factors  to  develop  the  substantive  areas  of  analysis? 

The  intelligence  issue  statement  developed  for  the  case  study  has  an  implicit  customer 
assumption:  the  Blues  will  attempt  to  overthrow  General  Alpha.  While  your  analysis  may 
ultimately  indicate  that  the  Blues  will  not  attempt  overthrow,  first  consider  what  would  be  the 
sequences  of  events  that  will  shape  a  Blue  decision  to  attack  General  Alpha?  From  a  Blue 
perspective,  what  would  they  consider  before  making  a  decision,  what  do  you  need  to  know 
about? 


•  The  status  of  the  Blues'  combat  capabilities.  Some  considerations  would 
be  potential  improvements,  in  the  short-term,  through  acquisition  of 
more  arms  and  training  and  by  recruiting  more  personnel. 
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•  An  assessment  by  the  Blues  of  the  combat  effectiveness  of  General 
Alpha's  forces.  Some  considerations  in  this  assessment  would  include 
equipment,  training,  recent  performances,  deployment,  and  loyalty  of 
the  troops. 

•  The  state  of  the  internal  political  environment  in  Orange.  Some 
considerations  are  the  popularity  of  General  Alpha,  the  standard  of 
living,  the  degree  the  citizens  are  oppressed,  the  effects  of  any 
efforts  to  change  General  Alpha's  popularity,  and  other  social  and 
economic  conditions  within  the  country. 

•  The  state  of  the  external  political  environment.  Some  considerations 
are  General  Alpha's  relationship  with  neighboring  countries  and  the 
degree  of  active  support  provided  to  General  Alpha. 

•  The  possibility  of  attempting  to  measure  current  conditions  by 
conducting  small  scale  conf licts  with  General  Alpha's  forces  and  then 
evaluating  the  results. 

•  The  acquisition  of  f  inancial  and  other  support  for  the  operation  of  the 
Blue  forces. 

•  Any  other  factors  that  would  inform  the  Blues  about  their  ability  to 
succeed. 

You  must  consider  a  second  aspect  of  this  intelligence  issue  while  developing  the  substantive 
areas  of  analysis.  That  is,  will  the  Blue  forces  succeed  if  they  decide  to  attack?  If  the  Blues 
forces  do  take  action  against  General  Alpha,  they  would  have  an  expectation  of  success.  The 
Blues  may  or  may  not  have  based  this  expectation  of  success  on  a  realistic  consideration  of  the 
factors  involved  in  attacking.  In  your  assessment  of  whether  or  not  the  Blues  will  succeed  if  they 
attack,  the  following  are  examples  of  some  events  you  would  consider: 

•  The  deployment  of  General  Alpha's  forces  and  the  General's  ability  to 
move  the  forces. 

•  The  history  of  how  General  Alpha  originally  seized  power  in  terms  of 
leadership,  tactics,  and  his  reaction  to  adverse  information. 

•  The  existence  of  support  from  leaders  of  other  countries. 

For  this  intelligence  issue  statement,  the  bullets  in  the  blue  different  font  provide  examples 
of  the  substantive  areas  associated  with  the  intelligence  issue.  These  will  be  listed  in  your 
analysis  plan,  along  with  the  information  described  later  in  this  section.  These  bullets  describe 
dynamics  and  interactions,  which  include  actors,  the  environment,  artifacts,  and  the  time  frame 
relevant  to  the  issue. 

In  addition  to  considering  all  the  factors  associated  with  Blues’  decision  to  attack,  you  would 
also  consider  the  sequences  of  decisions  and  events  associated  with  Gen  Alpha’s  possible 
preventative  actions  and  reactions  to  attack,  as  well  as  those  of  other  countries  (e.g.  Country 
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Green).  These  are  beyond  the  scope  of  this  example,  but  would  they  would  be  considered  during 
an  actual  intelligence  analysis  effort. 

To  continue  the  example,  the  acquisition  and  analysis  of  information  about  the  above  events  and 
factors  could  lead  you  to  produce  many  different  assessments.  Some  of  these  might  include: 

•  The  Blues  may  decide  not  to  attack  General  Alpha’s  forces 

•  The  Blues  may  decide  on  a  different  approach  to  overthrow  General  Alpha 

•  The  Blues  may  attack  and  achieve  an  easy  and  quick  victory 

•  The  Blues  may  be  decisively  defeated 

•  General  Alpha  may  lose  total  control  but  retain  enough  influence  so  a  prolonged 
civil  war  will  ensue 

The  result  will  determine  what  the  after  effects  will  be.  The  intelligence  issue  statement  also 
requires  your  assessment  of  the  consequences  of  failure  if  the  Blues  do  attack.  Since  all  the 
events  associated  with  the  after  effects  have  yet  to  occur,  little  direct  information  exists  so  an 
assessment  of  what  will  happen  has  to  be  based  on  knowledge  of  what  has  happened  in  similar 
circumstances  and  the  past  behavior  of  the  Blue  leaders  and  General  Alpha.  Substantive  areas 
associated  with  similar  circumstances  and  past  Blue  and  General  Alpha  behavior  would  also  be 
listed  in  your  analysis  plan. 

In  summary,  by  thinking  about  the  dynamics  and  interactions  (including  actors,  artifacts, 
environments,  and  time  frames)  associated  with  the  intelligence  issue,  you  can  begin  to  identify 
substantive  areas  needed  to  address  the  intelligence.  How  then  will  you  use  those  substantive 
areas  of  analysis  to  develop  the  analysis  plan? 

5.3.2.  List  the  Major  Milestones  for  the  Analysis  Effort 

When  you  used  the  Customer  View  Guide  to  develop  your  intelligence  issue  statement,  at  least 
two  of  the  questions  on  the  guide  were  intended  to  help  you  collect  information  about  the 
customer’s  time  frames  for  both  the  intelligence  product  and  the  decision  to  be  based  on  the 
product.  For  self-initiated  issues,  you  may  have  an  idea  of  when  your  project  will  be  needed  by 
prospective  customers  or  when  it  has  to  be  completed  so  that  you  can  work  on  other  efforts. 

AOR  issues  will  be  continuing,  but  within  your  AOR  issue  there  may  already  be  self-initiated 
issues  you  have  identified  to  address  within  a  certain  time  period  or  incoming  external  issues  that 
require  attention  by  a  given  date.  These  dates  can  be  included  in  your  AOR  issue  analysis  plan. 

All  of  these  dates  are  examples  of  milestones,  or  key  events  that  can  be  used  to  orient  planning 
for  the  intelligence  analysis  effort.  You  need  to  determine  what  the  milestones  are  for  your 
intelligence  analysis  effort  so  that  you  can  plan  your  work  and  coordinate  the  work  of  others,  if 
needed.  The  milestones  you  identify  during  this  step  should  be  major  events,  not  the  start  and 
stop  dates  for  every  step  in  the  intelligence  analysis  effort. 

5.3.3.  List  the  Activities  that  Must  Be  Completed  to  Address  the  Intelligence  Issue 

After  you  develop  the  substantive  areas  of  analysis,  you  can  begin  matching  who  has  the 
knowledge  needed  with  those  substantive  areas  of  the  intelligence  issue.  When  you  look  at  the 
knowledge  required  to  understand  the  dynamics  and  interactions  of  the  situation,  including  its 
actors,  artifacts,  environment,  and  time  frames,  you  may  find  that  some  of  the  knowledge 
required  for  addressing  the  intelligence  issue  lies  outside  the  scope  of  your  responsibility. 
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If  some  of  the  substantive  areas  needed  are  outside  the  scope  of  your  responsibility,  you  will 
need  to  collaborate  with  and  obtain  contributions  from  other  analysts.  You  will  need  to  identity 
who  has  knowledge  about  the  actors,  environment,  artifacts,  and  time  frames  contained  in  the 
substantive  areas  of  analysis.  What  will  be  the  nature  of  your  collaboration  and  what  will  you 
need  them  to  contribute? 

In  addition,  since  you  have  identified  the  major  milestones  of  the  analysis  effort,  you  can  also 
discuss  with  other  analysts  the  time  frame  in  which  you  need  their  contributions. 

5.3.4.  Identify  any  other  Resources  Needed 

In  some  cases  your  analysis  effort  may  require  using  resources  in  addition  to  the  analysis  work 
done  by  you  and  others.  Examples  include  the  use  of  contractual  support  for  specific  technical 
areas  of  analysis  or  perhaps  obtaining  specialized  software  for  your  use  to  develop  a  computer- 
based  model.  If  your  intelligence  issue  requires  these  types  of  resources,  you  should  identify 
them  in  your  analysis  plan.  You  should  also  identity  the  activities  associated  with  using  these 
resources. 

5.3.5.  Develop  an  Overall  Schedule 

Once  you  have  identified  milestones  or  key  events  for  the  intelligence  analysis  effort,  what  needs 
to  be  done,  and  who  needs  to  do  it,  you  are  ready  to  prepare  an  overall  schedule.  Of  course,  the 
nature  of  your  time  frames  and  schedule  will  be  different  for  an  AOR  issue.  For  an  AOR  issue, 
the  focus  should  be  on  identified  time  frames  during  which  you  want  to  achieve  an 
understanding  of  the  various  substantive  areas  associated  with  your  AOR.  Thinking  about  these 
substantive  areas  will  later  aid  you  as  you  develop  your  information  plan  for  your  AOR,  which  is 
covered  in  the  next  FAC  section. 

You  may  generate  a  manual  version  of  the  schedule  or  use  computerized  tools  to  assist  you  with 
the  schedule.  We  have  included  a  sample  Analysis  Plan  in  subsection  4.5.  However,  the  format 
you  choose  to  use  is  up  to  you.  It  may  simply  be  a  list  of  activities,  dates,  and  people  involved  or 
it  may  consist  of  a  graphical  representation  of  the  relationships  between  work  components,  time 
to  complete,  and  major  events.  This  graphical  representation  may  be  supplemented  with  notes  if 
special  considerations  are  involved  with  the  effort.  The  schedule  should  be  in  a  format  that  helps 
you  think  about  the  work  that  needs  to  be  accomplished  and  when  it  needs  to  be  completed.  The 
goal  of  this  step  is  to  produce  a  “useful,  visible  tool”  that  assists  you,  and  those  you  might  work 
with,  in  understanding  the  intelligence  analysis  effort. 

5.3.6.  Write  the  Analysis  Plan 

What  does  the  analysis  plan  cover?  A  sample  Analysis  Plan  using  the  case  study  information  in 
this  section  is  included  in  subsection  4.5.  A  typical  analysis  plan  should  contain  the  following: 

•  The  intelligence  issue  statement 

•  A  description  of  the  substantive  areas  of  analysis  for  the  intelligence  analysis  effort 
and  who  will  do  the  analysis  related  to  each  area 

•  A  schedule  for  the  activities  involved  with  the  intelligence  analysis  effort,  including 
review  activities 

•  The  resources  needed 
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•  Contingency  plans,  if  appropriate 

Since  the  analysis  plan  is  written  by  you,  for  you,  you  can  modify  the  contents  and  format  of  the 
analysis  plan  to  meet  the  needs  of  the  intelligence  effort  and  your  work  style. 

5.4  Summary 

The  importance  of  planning  for  a  wide  range  of  human  endeavors  is  well  understood  and  widely 
accepted.  Planning  for  an  intelligence  analysis  effort  can  present  some  unique  challenges,  but  the 
careful  consideration  of  the  substantive  areas  of  analysis  can  help  create  the  conditions  for  an 
effective  plan.  In  addition,  developing  the  substantive  areas  of  analysis  provides  a  foundation  for 
the  work  you  do  for  information  planning  and  acquisition,  the  topics  described  in  the  next  two 
sections  of  the  FAC. 

5.5  Analysis  Plan  for  the  Example  -  External  Issue 

INTELLIGENCE  ISSUE  STATEMENT  (24  May  09):  Understand  the  likelihood  that  the  Blues 
will  overthrow  General  Alpha  in  the  next  month  to  support  advice,  decision  making,  and 
planning  relative  to  providing  U.S.  assistance  to  the  Blues.  Understand  the  conditions  for  success 
and  failure  (with  evidence  supporting  each)  and  the  consequences  of  failure  (including  the 
rationale  for  assessing  those  consequences),  focusing  on  the  consequences  during  the  next  year. 

SUBSTANTIVE  AREAS  OF  ANALYSIS  (i.e.,  what  do  you  need  to  know  to  address  the 
intelligence  issue.  You  may  find  it  helpful  to  identify  those  areas  already  known  and  unknown.) 
INCLUDE  WHO  WILL  DO  THE  ANALYSIS  RELATED  TO  EACH  AREA: 

•  The  status  of  the  Blues  ’  combat  capabilities  (Known).  Some  considerations  would 
be  potential  improvements,  in  the  short-term,  through  acquisition  of  more  arms  and 
training  and  by  recruiting  more  personnel  (Largely  a  Known  but  need  to  update). 

>  Will  rely  on  finished  analysis  products  already  done  (by  me  and  another 
analyst  in  my  organization).  I  will  then  focus  analysis  efforts  on  incoming 
information  since  those  products  were  done. 

•  An  assessment  by  the  Blues  of  the  combat  effectiveness  of  General  Alpha’s  forces. 
Some  considerations  in  this  assessment  would  include  equipment,  training,  recent 
perfonnances,  deployment,  and  loyalty  of  the  troops  (Largely  a  Known  but  need  to 
update). 

>  Will  rely  on  finished  analysis  products  already  done  (by  me  and  another 
analyst  in  my  organization..  I  will  then  focus  analysis  efforts  on  incoming 
infonnation  since  those  products  were  done. 

•  The  state  of  the  internal  political  environment  in  Orange.  Some  considerations  are 
the  popularity  of  General  Alpha,  the  standard  of  living,  the  degree  the  citizens  are 
oppressed,  the  effects  of  any  efforts  to  change  General  Alpha’s  popularity,  and 
other  social  and  economic  conditions  within  the  country  (Knowns  and  unknowns, 
see  below). 

>  Will  rely  on  finished  analysis  products  done  by  CIA  -  Known.  Will  contact 
the  responsible  analyst  for  a  contribution  on  any  updates  in  perspective  since 
the  finished  product  was  done  -  Unknown. 
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The  state  of  the  external  political  environment.  Some  considerations  are  General 
Alpha’s  relationship  with  neighboring  countries  and  the  degree  of  active  support 
provided  to  General  Alpha  (Largely  aware  of,  mostly  know.  Need  to  update). 

>  Will  rely  on  finished  analysis  products  done  by  CIA.  Will  contact  the 
responsible  analyst  for  a  contribution  on  any  updates  in  perspective  since  the 
finished  product  was  done. 

The  possibility  of  attempting  to  measure  current  conditions  by  conducting  small 
scale  conflicts  with  General  Alpha’s  forces  and  then  evaluating  the  results. 

>  Will  review  recent  information,  discuss  with  other  analysts  and  solicit 
contributions  if  they  are  aware  of  any  planning  or  actions  -  Unknown. 

The  acquisition  of  financial  and  other  support  for  the  operation  of  the  Blue  forces 
(Largely  a  Known,  need  to  update). 

>  Will  rely  on  finished  analysis  products  done  by  Treasury.  Will  contact  the 
responsible  analyst  for  a  contribution  on  any  updates  in  perspective  since  the 
finished  product  was  done. 

Any  other  factors  that  would  infonn  the  Blues  about  their  ability  to  succeed 
(Unknowns,  need  to  spend  time  on). 

>  Review  finished  analysis  products  for  these  factors.  Discuss  with  key 
analysts  expert  on  Blue’s  situation.  Solicit  inputs  if  discussions  indicate  they 
will  be  of  use. 

The  deployment  of  General  Alpha’s  forces  and  the  General’s  ability  to  move  the 
forces  (Known,  but  need  to  update.) 

>  Will  rely  on  finished  analysis  products  already  done  (by  me  and  another 
analyst  in  my  organization.  I  will  then  focus  analysis  efforts  on  incoming 
infonnation  since  those  products  were  done. 

The  history  of  how  General  Alpha  originally  seized  power  in  terms  of  leadership, 
tactics,  and  his  reaction  to  adverse  information  (Known) 

>  Will  rely  on  finished  analysis  products  done  by  CIA  and  DIA.  Will  contact 
the  responsible  analysts  for  a  contribution  on  any  updates  in  perspective 
since  the  finished  products  were  done. 

The  existence  of  support  from  leaders  of  other  countries  (Largely  a  Known,  but  need 
to  update). 

>  Will  rely  on  finished  analysis  products  done  by  CIA  and  DIA.  Will  contact 
the  responsible  analysts  for  a  contribution  on  any  updates  in  perspective 
since  the  finished  products  were  done. 

SCHEDULE  OF  ACTIVITIES  (include  review  activities):  Based  on  the 
customer’s  request,  only  5  days  are  available  to  address  this  intelligence  issue.  As  a 
result,  all  contributions  must  be  received,  all  analysis  must  be  done,  and  all  review 
activities  must  be  done  within  the  5  days.  I  have  discussed  the  possibility  of  an 
abbreviated  review  cycle  with  management  and  they  have  agreed  to  Vi  day.  The 
planned  schedule  is  as  follows: 

>  Days  1  &  2,  request  contribution  initiate  discussions 
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>  Days  2,  plan  and  initiate  infonnation  acquisition 

>  Days  3  and  4,  receive  contributions 

>  Days  3  and  4,  do  analysis  and  begin  writing  the  product 

>  Day  5,  finish  the  product  and  complete  review 

RESOURCES  NEEDED:  Acquiring  information  will  require  some  assistance  from  our 
information  providers  in  the  library  function.  Efforts  will  be  made  to  have  priority  placed  on 
expediting  acquiring  info  for  this  effort.  No  contractual  or  other  resources  are  needed. 

CONTINGENCY  PLANS:  I  will  stay  in  contact  with  all  parties  involved  in  the  effort  to  try  and 
ensure  that  effort  stays  on  track,  given  the  extremely  short  time  frame. 
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6.0 


DEVELOPING  THE  INFORMATION  PLAN 


...the  only  man  to  do  the  collecting  of  data  (beyond  obvious  materials)  which  he  cannot 
name  by  title  is  the  one  who  knows  what  he  is  looking  for.  Should  there  exist  a  man  in  the 
library  unit  who  was  so  great  an  Iranian  expert  that  the  professional  Jones  could  make 
his  wants  known  without  giving  a  lecture  course  on  Iranian  life  and  politics,  then  this 
man  should  not  be  on  the  library  staff.  He  should  be  on  Mr.  Jones's  staff. 80 

6.1  Introduction 

At  this  point  in  the  FAC  you  have  developed  a  good  understanding  of  the  intelligence  issue  and 
identified  the  substantive  areas  of  intelligence  analysis.  You  have  planned  for  the  resources  that 
are  required,  identified  who  will  do  the  work,  and  when  the  work  needs  to  be  completed.  The 
next  stage  in  the  FAC  is  to  translate  your  understanding  of  the  intelligence  issue  and  the  events, 
participants,  and  decisions  described  by  the  substantive  areas  of  intelligence  analysis  into  a 
strategy  for  seeking  information  about  those  topics.  The  strategy  you  develop  for  seeking 
information  will  be  contained  in  an  information  plan,  a  written  document  that  describes  the 
categories  of  information  to  search,  where  to  look  for  information,  and  any  conditions  of  the 
search.  You  can  think  of  the  information  plan  as  being  a  protocol,  or  a  set  of  guidelines,  for 
locating  the  information  you  need  in  order  to  address  the  intelligence  issue. 

The  importance  of  developing  a  comprehensive  and  valid  infonnation  plan  cannot  be  overstated. 
Although  developing  an  information  plan  will  not  guarantee  a  high  quality  analytical  outcome, 
poor  information  acquisition  will  unquestionably  lead  to  a  poor  analytic  outcome.  The 
information  plan  is  an  essential  step  in  ensuring  you  will  be  successful  in  acquiring  as  much 
quality  infonnation  as  humanly  possible,  within  the  time  available,  in  order  to  provide  the  best 
possible  response  to  the  intelligence  issue. 

Keep  in  mind  that  your  information  plan  most  likely  will  not  be  static.  While  the  subject  areas 
needed  to  address  the  intelligence  issue  may  not  change,  the  acquisition  of  information  in  the 
subject  areas  will  be  anything  but  fixed.  As  you  acquire  information,  this  infonnation  may 
produce  more  avenues  for  exploration  and  analysis.  Much  of  this  activity  will  not  impact  the 
overall  infonnation  plan.  However,  after  you  acquire  some  information,  you  may  realize  that  you 
need  to  explore  a  new  path.  When  this  occurs  you  may,  of  necessity,  have  to  alter  the  original 
information  plan. 

Where  does  the  creation  of  the  information  plan  fit  into  the  analysis  process? 

In  the  Introduction  and  Background  Section  of  the  FAC  we  discussed  the  intelligence  analysis 
process  first  described  by  Sherman  Kent.  Kent’s  process  included  the  following  steps: 

2.  The  appearance  of  a  problem  requiring  the  attention  of  a  strategic  intelligence  staff. 

3.  Analysis  of  this  problem  to  discover  which  facets  of  it  are  of  actual  importance  to  the 
U.S.  and  which  of  several  lines  of  approach  are  most  likely  to  be  useful  to  its 
governmental  customers. 

4.  Collection  of  data  bearing  upon  the  problem  as  formulated  in  Stage  2.  This  involves  a 
survey  of  data  already  at  hand  and  available  in  the  libraries  of  documentary  materials, 


80  Kent,  S.  Strategic  intelligence  for  American  world  policy.  Princeton  University  Press,  Princeton,  NJ,  1949,  p.  137 

81  Kent  pp.  157-158. 
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and  an  endeavor  to  procure  new  data  to  fill  in  gaps. 

As  Kent  described  in  Step  2,  when  you  developed  the  intelligence  issue  statement,  you  began 
your  analysis  of  the  intelligence  problem.  You  identified  those  facets  of  the  issue  that  are  of 
importance  to  your  actual  or  prospective  customers.  Your  analysis  of  the  problem,  which  we 
refer  to  as  the  intelligence  issue,  continued  as  you  produced  the  analysis  plan.  By  further 
exploring  and  elaborating  the  substantive  knowledge  needed  to  address  your  intelligence  issue 
statement,  you  laid  the  groundwork  for  Step  3,  which  we  refer  to  as  information  acquisition  in 
the  FAC.  Before  you  can  begin  infonnation  acquisition,  you  have  to  identify  subject  areas  for 
searching  and  the  sources  to  be  searched.  Once  you  have  identified  subject  areas  and  sources  you 
have  created  what  we  call  an  acquisition  strategy,  this  acquisition  strategy  is  described  in  the 
information  plan. 

Who  is  the  audience  for  the  information  plan? 

The  information  plan  is  created  by  you,  for  you,  to  set  the  direction  for  information  seeking.  It  is 
another  instance  of  the  FAC’s  “writing  as  thinking”  emphasis.  Developing  an  acquisition 
strategy  is  a  major  cognitive  challenge;  the  goal  of  setting  a  direction  for  information 
acquisition  is  to  increase  the  effectiveness  of  your  infonnation  seeking  activities. 

Why  does  the  intelligence  analyst  develop  the  information  plan? 

The  quote  from  Kent  that  opened  this  section  of  the  FAC  is  still  applicable  today.  As  we 
discussed  in  the  Introduction  and  Background  Section,  the  meaning  of  information  is  dependent 
on  the  individual  making  the  assignment  of  meaning.  Who  better  knows  what  infonnation  is 
needed  to  solve  a  problem  than  the  person  who  has  to  solve  the  problem?  As  a  result,  you,  as  the 
analyst  with  the  expertise  needed  to  address  the  intelligence  issue,  play  an  essential  role  in 
establishing  what  information  is  needed  and  how  to  obtain  this  infonnation.  This  role  cannot  be 
delegated  to  others.  Y ou  may  consult  others  in  order  to  locate  possible  sources  for  information, 
to  understand  the  specifics  of  search  strategies  or  systems,  or  to  help  you  in  other  ways  with  your 
search.  However,  you  must  identify  the  acquisition  strategy  you  intend  to  follow  and  therefore 
set  the  direction  for  the  search. 

How  does  developing  an  information  plan  differ  from  establishing  Intelligence  Collection 
Requirements  (ICRs)? 

Developing  an  information  plan  for  an  intelligence  issue  is  not  the  same  as  establishing  ICRs  for 
the  different  infonnation  sources  or  “INTs”  (e.g.,  Imagery  Intelligence  (IMINT),  Measurement 
and  Signature  Intelligence  (MASINT),  or  Human  Intelligence  (HUMINT).  While  establishing 
ICRs  is  a  critical  part  of  your  role  as  an  intelligence  analyst,  analysis  for  an  intelligence  issue 
will  likely  be  based  on  information  that  is  already  collected.  If,  as  you  work  on  an  intelligence 
issue,  you  only  use  information  delivered  to  you  in  response  to  standing  or  ad  hoc  ICRs,  it  is 
possible  that  you  will  miss  critical  information  needed  to  address  an  intelligence  issue.  The 
information  plan  is  designed  to  address  a  specific  intelligence  issue  and  considers  all  information 
sources  as  possibilities  for  useful  information.  Unless  you  are  new  to  your  AOR  (i.e.,  working  an 
AOR  issue),  the  infonnation  plan  assumes  that  established  ICRs  for  your  area  of  responsibility 
are  already  in  place  and  being  updated.  If  you  are  working  on  an  AOR  issue,  you  probably  will 
need  to  establish  your  ICRs  and  review  any  that  might  already  be  in  place  for  your  AOR. 
Regardless  of  the  type  of  issue  you  working,  if  you  see  a  need  for  additional  or  modified  ICRs, 
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you  should  certainly  include  planning  to  do  them  in  your  information  plan.  But,  if  you  rely  solely 
on  ICRs  to  address  an  intelligence  issue,  you  risk  producing  a  flawed  analysis. 


JUST  A  NOTE  BEFORE  YOU  CREATE  AN  INFORMATION  PLAN 

Developing  an  information  plan  relies  on  knowledge  of  both  intelligence  community  and 
open  literature  sources.  For  the  purposes  of  the  FAC,  we  assume  that  intelligence  analysts 
have  at  least  a  working  knowledge  of  these  repositories  of  information.  If  you  do  not  have 
at  least  a  background  understanding  of  the  information  sources  available  to  you,  you  may 
want  to  consider  activities  that  would  improve  your  knowledge  of  these  information  sources 
before  you  create  an  information  plan.  You  may  find  that  the  Appendices  included  with  this 
section  will  be  helpful  not  only  as  you  create  a  specific  information  plan  but  as  you  learn 
more  about  the  information  sources  available  to  you. 


6.2  Building  the  Information  Plan 

Developing  an  information  plan  consists  of  developing  the  acquisition  strategy  you  intend  to 
follow  to  address  the  intelligence  issue.  Creating  the  information  plan  consists  of  the  following 
activities: 

•  Identity  broad  topics  for  information  acquisition,  based  on  the  substantive  areas  of 
analysis  identified  in  the  analysis  plan 

•  Identity  the  best  resources  for  locating  information  related  to  the  broad  topics 

•  Select  the  IS&R  systems  to  be  used  for  information  acquisition 

•  Identity  who  will  acquire  the  identified  information 

•  Create  the  schedule  for  acquiring  the  identified  information 

•  Write  the  information  plan 

6.2.1.  Identify  Broad  Topics  for  Information  Acquisition 

As  any  analyst  knows,  scarcity  of  information  on  a  broad  topic  is  not  usually  a  problem. 
Intelligence  collection  systems  acquire  an  ever  increasing  amount  of  data,  much  of  which  is 
neither  analyzed  nor  even  reviewed.  In  addition,  open  source  information  from  the  internet  and 
world  publications  grows  at  an  exponential  rate.  As  an  analyst  for  your  area  of  responsibility,  if 
you  establish  information  requirements  that  are  too  broad,  you  can  easily  be  overwhelmed  by  the 
amount  of  available  information.  Conversely,  if  you  are  overly  specific  in  expressing  your 
information  needs,  you  can  miss  information  essential  to  your  analytic  responsibilities. 

Balancing  the  breadth  and  the  specificity  of  your  infonnation  seeking  applies  to  your  response  to 
a  specific  intelligence  issue  just  as  it  does  to  your  overall  area  of  responsibility.  Establishing  a 
balance  between  these  two  extremes  of  information  retrieval  is  an  absolute  necessity  for  success 
in  addressing  your  intelligence  issue. 

When  you  created  the  analysis  plan  you  devoted  considerable  effort  to  developing  the 
substantive  areas  of  analysis  associated  with  your  intelligence  effort.  You  will  use  these 
substantive  areas  of  analysis  as  the  basis  for  identifying  the  broad  topics  for  which  you  will 
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search  during  information  acquisition.  These  search  areas  will  likely  consist  of  subject  matter 
content,  personalities,  organizations,  activities,  equipment,  and  locations  related  to  the 
intelligence  issue.  For  example,  you  may  have  identified  that  organization  XYZ  could  play  a 
major  role  in  the  decision  making  related  to  your  intelligence  issue.  As  a  result,  you  will  want  to 
seek  information  about  organization  XYZ  and  the  individuals  associated  with  it.  At  this  stage  of 
the  FAC,  you  do  not  need  to  identify  the  specific  individuals  of  potential  interest,  but  if  you 
already  know  their  names,  you  can  list  them.  Otherwise,  the  information  plan  will  include 
“organization  XYZ”  and  “individuals  associated  with  XYZ.”  Or,  you  may  want  to  get 
information  on  organizations  you  don’t  yet  know  about  by  searching  on  various  activities.  In  this 
case  your  information  plan  would  list  those  activities  as  search  areas. 

In  addition  to  identifying  key  topics,  personalities,  organizations,  activities,  equipment,  and 
locations  as  search  topics,  you  may  also  want  to  include: 

•  terminology  or  vocabulary  new  to  you 

•  desired  media  -  do  you  want  videos,  photos,  print,  or  other  types  of  data? 

The  end  result  of  characterizing  the  underlying  information  associated  with  the  substantive  areas 
of  analysis  is  a  list  of  search  topics  -  what  to  include  and  what  to  exclude  during  information 
acquisition. 

6.2.2.  Identify  the  Best  Resources  for  Locating  Information 

The  intelligence  community  operates  a  wide  variety  of  collection  programs  that  are  potential 
resources  during  infonnation  acquisition.  While  you  might  search  each  of  these  resources  for 
every  intelligence  issue  you  encounter,  it  would  not  be  effective  or  efficient  to  do  so.  In  order  to 
use  your  time  most  effectively,  you  should  determine  which  of  the  information  sources  available 
to  you  is  likely  to  provide  the  infonnation  you  need. 

Supplement  to  this  section  of  the  FAC  is  a  reference  that  lists  many  of  the  various  sources  (e.g., 
INTs)  you  might  consider  for  an  intelligence  issue.  Other  sources  of  infonnation  may  exist  for 
your  intelligence  issue;  add  them  to  the  list  as  needed.  Using  your  knowledge  of  the 
characteristics  and  content  of  the  information  from  these  sources  will  help  you  decide  whether  or 
not  each  information  source  may  or  may  not  be  of  use.  If  you  see  a  source  that  you  have  not  used 
previously,  but  which  seems  to  apply  to  your  intelligence  issue,  consult  with  an  information 
specialist  or  fellow  analysts  who  have  experience  with  this  type  of  information  source  for  more 
information  about  it.  Once  you  know  more  about  a  source  you  can  decide  whether  or  not  you 
will  use  it  for  your  intelligence  issue.  Use  the  comments  column  to  record  your  impressions  of 
each  source  and  to  identify  those  sources  which  seem  to  be  most  relevant  to  your  acquisition 
strategy.  You  will  use  these  comments  when  you  create  the  schedule  for  information  acquisition 
and  the  overall  information  plan. 

6.2.3.  Select  IS&R  Systems 

After  you  identify  the  information  sources  (Supplement  1)  that  are  related  to  the  search  topics 
you  must  then  detennine  which  IS&R  systems  contain  information  from  these  sources.  This  step 
would  be  easier  if  the  following  recommendation  had  been  implemented: 
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Create  an  IC-wide  information  repository.  Analysts  should  be  able  to  go  to  one  place  to 

82 

find  out  what  information  is  available  across  the  IC  and  how  they  can  access  it. 

Unfortunately,  no  such  intelligence  community  repository  or  IS&R  system  exists.  Instead,  an 
information  source  may  have  an  individual  IS&R  system  established  for  it.  In  addition,  many 
IS&R  systems  were  implemented  to  serve  the  needs  of  a  single  organization.  To  further 
compound  the  task  of  obtaining  the  all  the  infonnation  relevant  to  the  issue,  the  established 
IS&R  systems  are  not  exhaustive;  that  is,  they  do  not  contain  all  the  available  infonnation  from  a 
particular  information  source.  Finally,  the  content  of  one  IS&R  system  may  be  duplicated  in 
other  IS&R  systems. 

Naturally  you  want  to  ensure  that  you  acquire  all  relevant  information  from  a  given  information 
source.  However,  the  potential  exists  that  if  you  use  all  the  IS&R  systems  that  contain  a  specific 
information  source,  you  will  get  the  same  information  multiple  times.  Receiving  duplicate 
information  does  increase  the  amount  of  time  you  expend  to  review  the  search  results.  On  the 
other  hand,  if  you  try  to  avoid  this  problem  by  using  only  one  IS&R  system,  the  result  may  be  an 
incomplete  coverage  of  the  available  relevant  infonnation.  This  dilemma  requires  that  you  have 
knowledge  of  the  scope  and  coverage  of  each  of  the  IS&R  systems  operating  in  the  intelligence 
community  so  that  you  can  determine  which  ones  should  be  used  to  obtain  the  best  coverage 
while  minimizing  duplication.  How  do  you  acquire  data  about  the  scope  and  coverage  of  each  of 
these  systems? 

Supplement  2  to  this  section  of  the  FAC  is  a  guide  to  aid  you  in  characterizing  each  of  the  IS&R 
systems  you  might  consider  for  an  intelligence  issue.  Knowing  who  controls  access  and  how  you 
get  approval  for  that  access,  the  classification  level  of  the  content  of  the  IS&R,  how  the  system  is 
accessed,  the  form  of  the  IS&R  systems  output,  and  whether  you  are  familiar  with  the  retrieval 
language  are  all  aspects  of  an  IS&R  system  that  help  you  with  plans  for  using  it  if  you  detennine 
that  it  is  relevant  to  your  search.  Some  factors,  such  as  obtaining  approval  for  access,  may  also 
affect  the  schedule  you  create  for  information  acquisition. 

A  Special  Note  about  Open  Source: 

An  important  element  in  this  whole  question  of  data  collection  planning  is  that  the  analyst  must 
constantly  be  on  guard  to  avoid  what  could  be  thought  of  as  the  official  data  only  syndrome.  This 
manifests  itself  in  intelligence  units  where  the  prevailing  culture  is  one  in  which  trust  is  vested  in 
official  sources  only;  all  others  are,  to  a  degree,  regarded  with  uncertainty,  concern  or 

mistrust 83 

Although  the  need  for  a  single  comprehensive  open  source  IS&R  system  was  identified  in 
1995,  such  a  comprehensive  system  does  not  currently  exist  for  the  intelligence  community. 
The  intelligence  community  does  have  access  to  a  wide  variety  of  databases  of  open  source 
information.  Agencies  may  select  and  process  information  deemed  of  interest  to  the  intelligence 
community  but  not  otherwise  readily  available.  Much  of  this  information  has  undergone 
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translation  by  an  agency  or  its  contractors  and  is  stored  in  the  agency’s  IS&R  system.  In 
addition,  your  organization  or  the  broader  intelligence  community  may  provide  you  with  access 
to  commercially  available  open  source  infonnation  for  which  they  pay  a  subscription  fee. 

An  overwhelming  amount  of  open  source  infonnation  exists  in  repositories  not  associated  with 
intelligence  IS&R  systems.  With  the  advent  of  the  internet,  a  plethora  of  information  is  available; 
the  growth  of  the  internet  shows  no  signs  of  abating.  According  to  one  author  there  are 
approximately  20  billion  “documents”  in  cyberspace,  also  known  as  the  visible  web.  The  visible 
web  is  infonnation  that  can  be  accessed  through  the  use  of  a  search  engine,  such  as  Google, 

Bing,  Yahoo,  Alta  Vista,  HotBot,  MSN,  Ask,  or  Firefox. 

The  estimate  above  does  not  include  a  further  550  billion  documents  that  reside  on  the  deep  or 
invisible  Web.  The  invisible  Web  has  been  described  as  publicly  accessible,  non-proprietary 
pages  that  are  not  ’seen’  by  the  spiders  of  general  search  engines.  However,  search  engines  are 

85 

working  on  finding  ways  to  make  this  material  accessible. 

While  many  view  the  internet/world-wide  web  as  “readily  accessible  and  free,”  this  is  not  true  of 
all  sites  on  the  internet  since  much  of  the  deep  Web  is  not  readily  accessible.  Whichever  search 
engine  is  used,  no  search  engine  is  able  to  search  and  index  the  entire  internet.  This  includes 
even  the  visible  web  since  some  sites  will  deny  access  to  the  tools  used  by  these  services.  Some 
sites  also  allow  individuals  to  access  content  by  directly  browsing  the  web  site,  but  do  not  allow 
their  content  to  be  searched  by  general  search  engines. 

We  discuss  all  of  these  situations  with  the  internet  so  that  you  understand  that  a  single  approach 
to  searching  the  internet  may  result  in  incomplete  or  unhelpful  results.  You  must  familiarize 
yourself  with  all  of  the  likely  ways  that  you  might  use  to  search  on  the  web.  At  the  same  time, 
you  will  have  the  dilemma  we  discussed  earlier:  How  can  you  find  relevant  information  without 
an  overwhelming  expenditure  of  time  and  resources? 

As  a  result  of  the  above  situation,  identifying  which  open  source  IS&R  systems  are  likely  to 
contain  the  information  you  need  requires  an  extensive  knowledge  of  each  of  them.  If  your 
organization  has  established  personnel  who  serve  as  information  providers,  they  can  be  of 
assistance  since  awareness  and  knowledge  of  the  content  and  operational  characteristics  of  IS&R 
systems  is  part  of  their  responsibility.  The  information  provider  is  an  invaluable  source  for 
acquiring  the  information  needed  to  complete  the  guide  in  Supplement  2. 

6.2.4.  Identify  Who  Will  Acquire  the  Identified  Information 

Y ou  may  not  be  the  only  person  involved  in  information  seeking  for  a  particular  intelligence 
issue.  Some  analysts  rely  on  search  intermediaries  such  as  organizational  information  providers 
to  actually  conduct  online  or  hard  copy  searches  for  information.  Some  IS&R  systems  may  be 
complex  or  unique  and  require  searchers  with  specialized  knowledge  in  order  to  obtain  useful 
results.  In  order  to  develop  your  acquisition  strategy  completely,  we  suggest  that  you  consider 
each  information  source  you  plan  to  use.  Determine  who  will  actually  conduct  the  infonnation 
acquisition  activity  for  that  source,  including  specific  names  if  available.  You  will  need  this 
information  in  order  to  create  the  schedule  for  the  information  plan. 
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6.2.5.  Create  an  Information  Acquisition  Schedule 

At  this  stage  you  have  the  following  inputs  to  your  acquisition  strategy: 

•  The  broad  search  topics  for  information  relevant  to  the  intelligence  issue 

•  Your  judgment  on  the  potential  for  each  information  source  (INTs)  to  provide 
relevant  information 

•  A  determination  of  which  IS&R  systems  contain  the  desired  INT  information 

•  Who  will  do  the  actual  acquisition  of  the  information  about  each  search  topic 

•  The  time  allocated  in  the  analysis  plan  to  information  acquisition 

These  inputs  allow  you  to  make  a  series  of  judgments.  While  the  ideal  scenario  in  information 
acquisition  would  be  for  you  to  search  until  you  have  located  all  of  the  possible  relevant 
information,  operational  realities  usually  do  not  allow  for  this  ideal.  How  do  you  create  a 
realistic  schedule  for  information  acquisition? 

The  most  critical  input  to  creating  an  information  acquisition  schedule  is  the  time  allocated  to  it 
in  the  overall  schedule  developed  for  the  analysis  plan.  Given  that  timeframe,  what  are  the 
specific  actions  that  are  likely  to  result  in  the  retrieval  of  the  maximum  amount  of  relevant 
information  in  the  time  allotted?  In  order  to  help  you  determine  those  actions,  the  following 
paragraphs  contain  some  considerations  and  hints.  Of  course,  you  may  find  that,  upon  closer 
examination,  you  need  to  adjust  the  time  allocated  for  infonnation  acquisition  in  the  analysis 
plan.  You  may  make  this  adjustment  if  the  overall  schedule  permits  it. 

We  recommended  earlier  in  this  section  that  you  should  note  those  sources  in  Supplement  1 
which  are  the  richest  in  the  amount  of  relevant  information  or  potential  value  of  the  information. 
These  are  now  inputs  to  your  judgments  about  the  schedule. 

Then,  in  Supplement  2,  you  collected  information  on  where  such  information  might  be  found  and 
a  variety  of  factors  that  will  govern  the  ease  and  timeliness  of  using  such  systems.  As  you  create 
the  schedule  you  need  to  consider  factors  such  as: 

•  Do  you  maintain  a  personal  collection  of  information  on  the  intelligence  issue? 

How  current  is  your  personal  collection  of  information?  What  is  your  familiarity 
with  the  topic?  If  you  plan  to  search  your  own  holdings  on  a  topic  as  well  as  the 
sources  identified  in  Supplement  2,  you  need  to  allocate  time  to  that  activity  in  the 
infonnation  plan. 

•  How  critical  are  various  information  topics  to  the  overall  infonnation  acquisition 
effort?  If  the  schedule  to  address  your  issue  is  limited,  you  may  need  to  make 
tradeoffs  relative  to  search  areas  and  IS&R  systems  pursued.  This  may  be  quite 
difficult  to  detennine  before  searching  begins,  but  is  something  you  may  need  to 
consider  initially  and  later  when  you  are  searching. 

•  Do  you  already  have  access  to  the  selected  IS&R  system  or  do  you  need  to  obtain  it 
before  you  can  conduct  a  search?  The  time  needed  to  identify  who  controls  access 
and  then  obtaining  access  needs  to  be  considered  in  the  schedule. 

•  Do  you  know  how  to  use  the  IS&R  system?  Your  schedule  needs  to  consider  the 
time  it  would  take  to  leam  at  least  the  basics  of  using  the  IS&R  system  if  you  aren’t 
already  familiar  with  it.  An  experienced  user  can  operate  with  shorter  deadlines 
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than  someone  who  is  just  learning  the  system.  Your  schedule  should  reflect  your 
expertise  with  the  IS&R  systems  you  are  planning  to  use. 

•  Are  you  planning  on  working  with  intermediaries,  e.g.  infonnation  specialists, 
librarians,  or  other  information  providers,  during  infonnation  acquisition?  If  so,  you 
need  to  consider  their  availability  and  timeframes  in  your  schedule. 

•  Are  you  assuming  that  using  an  intennediary  will  shorten  the  time  needed  to 
complete  information  acquisition?  Using  an  intennediary  may  or  may  not  compress 
the  schedule  you  are  planning.  You  should  realistically  consider  such  factors  as 
communication  lags,  learning  periods,  and  interaction  during  infonnation 
acquisition  while  creating  a  schedule  that  includes  intermediaries. 

•  Have  you  considered  the  possibility  of  additional  information  acquisition  after  your 
initial  efforts  to  obtain  information?  You  may  find  that  your  analysis  of  the 
retrieved  information  indicates  that  other  sources  or  content  need  to  be  explored. 
Building  some  slack  in  the  schedule  to  accommodate  these  changes  in  acquisition 
strategy  is  a  good  idea  if  your  schedule  will  allow  it. 

Based  on  the  above  inputs,  you  can  create  a  timeline  for  information  acquisition  that  achieves  the 
coverage  we  described  earlier.  Your  schedule  will  optimize  your  efforts  to  obtain  the  maximum 
amount  of  relevant  information  while  still  meeting  your  or  the  customer’s  deadlines. 

6.2.6.  Write  the  Information  Plan 

The  information  plan  provides  the  direction  for  information  acquisition.  We  recommend  that  an 
information  plan  contains  the  following  sections: 

•  The  intelligence  issue  statement 

•  A  full  restatement  of  each  of  the  substantive  areas  of  intelligence  analysis  from  the 
analysis  plan 

•  The  acquisition  strategy  for  your  intelligence  issue: 

>  The  broad  topics  within  each  substantive  area  of  intelligence  analysis 
identified  in  the  analysis  plan 

>  The  sources  that  provide  information  of  the  desired  content  for  the  overall 
intelligence  issue 

>  Who  will  acquire  the  desired  infonnation  for  the  intelligence  issue 

•  An  overall  information  acquisition  schedule  that  summarizes  the  information 
acquisition  strategy  for  your  intelligence  issue  in  a  format  that  makes  sense  to  you 

Since  the  information  plan  is  written  by  you,  for  you,  you  can  create  an  information  plan  that 
suits  your  work  style  and  the  scope  and  complexity  of  the  intelligence  issue.  For  example,  you 
might  consider  basing  the  schedule  on  what  needs  to  be  done  for  each  of  the  substantive  areas  of 
intelligence  analysis.  You  might  also  develop  a  schedule  that  is  organized  by  IS&R  system. 
Choose  the  approach  that  best  supports  the  people  and  needs  of  your  current  intelligence  analysis 
effort. 

Subsection  5.6  provides  an  example  of  an  information  plan.  It  is  based  on  the  same  example  we 
discussed  for  the  “Developing  Intelligence  Issue  Statements”  and  the  “Setting  a  Direction:  The 
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Analysis  Plan”  sections  of  the  FAC.  We  included  enough  detail  in  the  example  to  illustrate  the 
information  plan,  but  we  did  limit  some  aspects  so  it  would  work  well  as  a  concise  example. 

6.3  Summary 

Whether  your  intelligence  issue  involves  a  short  turnaround  response  or  extensive  strategic 
analysis,  gathering  information  is  a  key  part  of  the  intelligence  analysis  process.  Setting  an 
acquisition  strategy  helps  you  address  an  issue  efficiently  and  effectively,  reduces  repetition  and 
mistakes,  and  allows  you  to  communicate  with  other  analysts  and  information  intermediaries. 

An  acquisition  strategy  also  helps  you  avoid  tunnel  vision  when  considering  possible  sources  of 
information.  If  you  systematically  consider  all  possible  sources,  you  minimize  the  risk  of  relying 
too  heavily  on  one  source  or  of  missing  relevant  sources  that  you  may  not  have  considered 
previously.  The  information  environment  that  you  are  expected  to  understand  and  use  is  rapidly 
expanding,  divergent,  and  anarchic  throughout.  An  information  plan  helps  you  navigate  this 
ever-changing  and  admittedly  difficult  landscape. 
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6.4 _ Supplement  1:  Information  Sources 


NOTE: 

The  sources  listed  are  a  starting  point.  Add  other  sources  as  needed  for  your  intelligence  issue. 


Information  Source 

Of  Use 

May  be 
of  Use 

Not  of  Use 

Comments 

Geospatial  Intelligence  (GEOINT)  Sources 

Electro-optical  imageiy 

Foreign  maps  &  charts 

Geomag  survey 

Hydro  survey 

Hyperspectral  imagery 

Infrared  imagery 

Light  detection  and  ranging  (LIDAR) 

Interferometric  synthetic  aperture  radar  (ISAR) 

Moving  target  indicator  (MTI) 

Multispectral  imagery  (MSI) 

Non-image  infrared 
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Information  Source 

Of  Use 

May  be 
of  Use 

Not  of  Use 

Comments 

Signal  Intelligence  (SIGINT)  Sources 

Communication  Intelligence  (COMINT) 

Electronic  Intelligence  (ELINT) 

Foreign  Instrumentation  Signal  Intelligence  (FISINT) 

HUMINT 

Other  Open  Source  Intelligence  (OSINT)  Sources 

Print 

Digital 

Radio 

Television 

Visible  Web 

Invisible  Web 
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Information  Source 

Of  Use 

May  be 
of  Use 

Not  of  Use 

Comments 

OSINT  Sources 

Finished  Intelligence  Products 
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6.5  Supplement  2:  IS&R  Systems 


IS&R 

Name 

Access 

Control 

Access 

Approval 

Classification 

Level 

How 

Accessed 

Output 

Form 

Used 

Previously? 

Source 

Content 
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6.6  Information  Plan  -  Example 

INTELLIGENCE  ISSUE  STATEMENT  (24  May  09):  Assess  the  likelihood  that  the  Blues  will 
overthrow  Gen  Alpha  in  the  next  month  to  support  advice,  decision  making,  and  planning 
relative  to  providing  U.S.  assistance  to  the  Blues.  Assess  the  conditions  for  success  and  failure 
(with  evidence  supporting  each)  and  the  consequences  of  failure  (including  the  rationale  for 
assessing  those  consequences),  focusing  on  the  consequences  during  the  next  year.  The  product 
was  due  NLT  3 1  May  09. 

SUBSTANTIVE  AREAS  OF  ANALYSIS 

•  The  status  of  the  Blues’  combat  capabilities.  Some  considerations  would  be 
potential  improvements,  in  the  short-term,  through  acquisition  of  more  arms  and 
training  and  by  recruiting  more  personnel. 

•  An  assessment  by  the  Blues  of  the  combat  effectiveness  of  General  Alpha’s  forces. 
Some  considerations  in  this  assessment  would  include  equipment,  training,  recent 
perfonnances,  deployment  and  loyalty  of  the  troops. 

•  The  state  of  the  internal  political  environment  in  Orange.  Some  considerations  are 
the  popularity  of  General  Alpha,  the  standard  of  living,  and  the  degree  the  citizens 
are  oppressed,  and  the  effects  of  any  efforts  to  change  General  Alpha’s  popularity, 
and  other  social  and  economic  conditions  within  the  country. 

•  The  state  of  the  external  political  environment.  Some  considerations  are  General 
Alpha’s  relationship  with  neighboring  countries  and  the  degree  of  active  support 
provided  to  General  Alpha. 

•  The  possibility  of  attempting  to  measure  current  conditions  by  conducting  small 
scale  conflicts  with  General  Alpha’s  forces  and  then  evaluating  the  results. 

•  The  acquisition  of  financial  and  other  support  for  the  operation  of  the  Blue  forces. 

•  Any  other  factors  that  would  inform  the  Blues  about  their  ability  to  succeed. 

•  The  deployment  of  General  Alpha’s  forces  and  the  General’s  ability  to  move  the 
forces. 

•  The  history  of  how  General  Alpha  originally  seized  power  in  terms  of  leadership, 
tactics,  and  his  reaction  to  adverse  information. 

•  The  existence  of  support  from  leaders  of  other  countries. 

TOPICS  ON  WHICH  INFORMATION  WILL  BE  SOUGHT 

As  was  pointed  out  in  the  Analysis  Plan  Section,  the  substantive  areas  relate  to  the  Blues’ 
decision  to  attack  General  Alpha  and  an  assessment  of  whether  or  not  the  Blues  will  succeed  if 
they  attack.  The  following  infonnation  is  relevant  to  making  judgments  on  the  substantive  areas 
identified. 

•  Blue’s  combat  capabilities 

>  Manning  levels 

>  Experience  levels 

>  Equipment 

>  Training 

105 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 

88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


>  Morale 

>  Performance 

>  Deployment 

•  General  Alpha’s  combat  capability 

>  Manning  levels 

>  Experience  levels 

>  Equipment 

>  Training 

>  Morale 

>  Performance 

>  Deployment 

•  Political  environment 

>  Form  of  government  and  stability  prior  to  General  Alpha’s  take  over 

>  Standard  of  living 

>  Freedom  of  speech 

>  Toleration  of  dissent 

>  Popular  support 

>  Propaganda  efforts 

•  International  support  -  General  Alpha 

>  Attitude  of  neighboring  countries 

>  Membership  in  regionaEintemational  forum 

>  Military  and/or  economic  aid 

•  International  support  -  Blues 

>  Attitude  of  neighboring  countries 

>  Military  and/or  economic  aid 

•  Personalities 

>  Blue  military  leadership 

>  General  Alpha 

>  Orange  government  leadership 

>  Orange  military  leadership 
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6.7  Supplement  1:  Information  Sources  (Example) 


NOTE: 

The  sources  listed  are  a  starting  point.  Add  other  sources  as  needed  for  your  intelligence  issue. 


Information  Source 

Of  Use 

May  be 
of  Use 

Not  of  Use 

Comments 

GEOINT  Sources 

Electro-optical  imagery 

X 

Foreign  maps  &  charts 

X 

Geomag  survey 

X 

Hydro  survey 

X 

Hyperspectral  imagery 

X 

Infrared  imagery 

X 

Light  detection  and  ranging  (LIDAR) 

X 

Interferometric  synthetic  aperture  radar  (ISAR) 

X 

Moving  target  indicator  (MTI) 

X 

Multispectral  imagery  (MSI) 

X 

Non-image  infrared 

X 
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Information  Source 

Of  Use 

May  be 
of  Use 

Not  of  Use 

Comments 

SIGINT  Sources 

COMINT 

X 

ELINT 

X 

FISINT 

X 

HUMINT 

X 

OSINT  Sources 

Print 

X 

Digital 

X 

Radio 

X 

Television 

X 

Visible  Web 

X 

Invisible  Web 

X 
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Information  Source 

Of  Use 

May  be 
of  Use 

Not  of  Use 

Comments 

Other  OSINT  Sources 

Finished  Intelligence  Products 

X 
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6.8  Supplement  2:  IS&R  Systems  (Example) 


NOTE: 

The  sources  listed  are  a  starting  point.  Add  other  sources  as  needed  for  your  intelligence  issue. 


IS&R 

Name 

Access 

Control 

Access 

Approval 

Classification 

Level 

How 

Accessed 

Output 

Form 

Used 

Previously? 

Source 

Content 

Library  of  National 
Intelligence 

Director  of 

National 

Intelligence 

Agency 

(DNI) 

Online 

Top 

Secret/Sensitive 

Compartmented 

Information 

(TS/SCI) 

Online 

Digital 

Yes 

Finished 

Intelligence 

DIA  HUMINT 

DIA 

Online 

Secret 

Online 

Digital 

Yes 

HUMINT 

CIA  WIRe 

CIA 

Online 

TS/SCI 

Online 

Digital 

No 

All  Source 

SIGINT 

NS  A 

Online 

TS/SCI 

Online 

Digital 

Yes 

COMINT 

Internet 

None 

N/A 

Unclassified 

Online 

Digital 

Yes 

Open  Source 

STRATFOR 

Local 

information 

providers 

Request 

Unclassified 

Intermediary 

Digital 

No 

Hidden  Web 
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6.9  Schedule  Associated  With  Information  Acquisition  (Example) 

Based  on  the  tasking,  5  days  are  available  to  provide  a  response  to  this  intelligence  issue.  As  a 
result,  only  1  day  can  be  allocated  to  do  information  acquisition.  The  acquisition  strategy  will 
make  the  maximum  use  of  finished  intelligence  products  coupled  with  academic  reporting 
available  on  the  Internet,  all  of  which  deal  with  the  topics  of  interest.  I’ll  contact  the  local 
information  providers  immediately  for  their  assistance  on  the  academic  and  STRATFOR 
information.  I  will  use  the  IS&R  systems  containing  recent  HUMINT  and  SIGINT  reporting  to 
ascertain  if  there  have  been  any  changes  from  the  status  described  in  the  finished  intelligence 
products.  A  few  hours  will  be  reserved  midway  through  the  analysis  so  that  additional 
information  acquisition  can  occur  based  on  the  analysis  of  the  initial  infonnation. 
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7.0 

7.1 


INFORMATION  ACQUISITION 
Introduction 


Now  that  you  have  developed  an  information  plan,  it  is  time  to  put  that  plan  into  action. 
Information  acquisition  is  the  process  of  identifying  and  obtaining  infonnation  from  the  various 
sources  you  identified  in  the  information  plan.  Infonnation  acquisition  provides  the  raw  material 
for  evaluation  and  study  so  that  you  can  assign  meaning  to  it,  marshal  evidence,  form  and 
evaluate  hypotheses,  and  develop  assessments  for  use  by  you,  other  analysts,  and  your 
customers. 

Where  does  information  acquisition  fit  into  the  analysis  process? 

In  several  previous  FAC  sections,  we  discussed  the  intelligence  analysis  process  first  described 
by  Sherman  Kent.  As  you  may  recall,  Kent’s  process  included  the  following  steps: 

3.  The  appearance  of  a  problem  requiring  the  attention  of  a  strategic  intelligence  staff. 

4.  Analysis  of  this  problem  to  discover  which  facets  of  it  are  of  actual  importance  to  the 
U.S.  and  which  of  several  lines  of  approach  are  most  likely  to  be  useful  to  its 
governmental  customers. 

5.  Collection  of  data  bearing  upon  the  problem  as  formulated  in  Stage  2.  This  involves  a 
survey  of  data  already  at  hand  and  available  in  the  libraries  of  documentary  materials, 
and  an  endeavor  to  procure  new  data  to  fill  in  gaps. 

As  Kent  described  in  step  2,  when  you  developed  the  intelligence  issue  statement,  you  began 
your  analysis  of  the  intelligence  problem.  You  identified  those  facets  of  the  issue  that  are  of 
importance  to  your  actual  or  prospective  customers.  Your  analysis  of  the  problem,  which  we 
refer  to  as  the  intelligence  issue,  continued  as  you  produced  the  analysis  plan.  By  further 
exploring  and  elaborating  the  substantive  knowledge  needed  to  address  your  intelligence  issue 
statement,  you  laid  the  groundwork  for  step  3,  which  we  refer  to  as  information  acquisition  in  the 
FAC. 

In  the  previous  section  of  the  FAC,  you  identified  subject  areas  for  searching,  the  sources  to  be 
searched,  and  created  an  acquisition  strategy.  In  this  section  of  the  FAC,  we  discuss  the  steps 
you  need  to  take  in  order  to  complete  the  work  described  in  Step  3. 

Developing  useful  and  complete  search  strategies  and  evaluating  the  results  of  a  search  are 

major  cognitive  challenges  for  the  analyst  during  information  acquisition.  It  is  essential  that  you 
are  actively  involved  in  information  acquisition  to  assure  that  the  direction  you  set  in  the 
information  plan  is  realized.  You  play  an  important  role  in  developing  search  strategies;  you  are 
irreplaceable  in  determining  whether  the  infonnation  retrieved  during  information  acquisition  is 
pertinent  to  the  intelligence  issue. 

7.2  Relevance 

Before  we  discuss  how  to  develop  a  search  strategy  and  retrieve  information,  we  need  to  discuss 
one  concept  in  depth.  In  order  to  be  useful  during  analysis,  the  information  you  obtain  as  a  result 
of  a  search  must  be  relevant.  What  is  relevant  information?  The  information  that  you  find  that 
pertains  to  your  intelligence  issue  is  relevant. 


S7  Kent,  S.  Strategic  Intelligence  for  American  World  Policy,  Princeton  University  Press,  Princeton,  NJ,  1949,  pp. 
157-158. 
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Another  key  characteristic  of  relevance  is  that  it  can  only  be  determined  by  the  person  who 
requested  the  information.  Information  is  either  accepted  or  rejected  based  on  the  judgment  of 
the  person  with  the  information  need.  You,  as  the  intelligence  analyst,  assess  relevance  during 
information  acquisition. 

The  goal  of  your  relevance  judgments  is  to  assure  that  your  search  strategy  and  retrieval 
processes  are  effective  and  that  the  data  you  acquire  is  potentially  useful  for  addressing  the 
intelligence  issue.  That  is,  the  relevance  judgment  you  are  making  answers  the  questions  “Is  this 
the  sort  of  thing  I  am  looking  for?  Does  it  pertain  to  the  intelligence  issue?”  You  are  making  a 
broad  judgment;  you  are  not  trying  to  evaluate  the  specifics  of  each  result  you  retrieve.  Instead, 
you  are  determining  whether  your  search  is  producing  results  that  are  topically  related  to  the 
intelligence  issue.  At  this  point,  your  focus  is  to  make  decisions  about  whether  or  not  to  retain 
the  information  you  find  for  further  consideration  in  your  analysis. 

The  initial  information  acquisition  activities  we  describe  in  this  section  of  the  FAC  are  only  the 
first  of  what  may  be  several  information  seeking  events  while  you  are  addressing  the  intelligence 
issue.  During  later  sections  of  the  FAC  we’ll  discuss  how  you  will  make  judgments  to  determine 
the  value  of  information,  assign  meaning  to  the  information,  evaluate  the  accumulated  evidence, 
and  formulate  and  evaluate  hypotheses  about  the  intelligence  issue.  As  you  make  these  more 
involved  judgments,  you  will  engage  in  additional  information  acquisition. 

7.3  Information  Acquisition 

Infonnation  acquisition  consists  of  the  following  activities: 

•  Formulation  of  an  initial  search  strategy  for  the  broad  topics  identified  in  the 
infonnation  plan 

•  Execution  of  the  search  strategy  in  each  IS&R  system  selected  in  the  information 
plan.  Execution  of  the  search  strategy  includes: 

>  Examination  of  the  results  of  each  search  to  determine  whether  the 
references  obtained  are  broadly  relevant 

>  Modification  of  the  search  strategy  to  incorporate  new  terms,  new 
resources,  or  to  change  details  of  the  search 

>  Repetition  of  the  above  steps  until  you  determine  that  the  search  is  complete 

•  Acquiring  and  managing  the  materials  retrieved  during  searching 


A  NOTE  ABOUT  INFORMATION  ACQUISITION 

The  guidelines  and  processes  we  describe  in  the  FAC  for  the  formulation  and 
execution  of  a  search  strategy  are,  by  necessity,  based  on  general  principles.  It  is 
essential  that  you  apply  these  principles  using  the  specifics  of  each  IS&R  system  you 
identified  in  the  information  plan.  We  strongly  suggest  that  you  work  with  information 
providers,  help  desks,  or  IS&R  documentation  while  you  are  constructing  and  carrying 
out  your  search  to  make  sure  that  you  take  full  advantage  of  the  features  and  contents 
of  the  IS&R  systems  you  plan  to  use. 
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7.3.1.  Formulation  of  an  Initial  Search  Strategy 

Information  acquisition  would  be  comparatively  simple  if  everyone  used  the  same  words  and 
phrases  to  express  infonnation,  ideas,  and  concepts.  Unfortunately  originality  and  variety,  rather 
than  conformity,  are  more  important  to  most  authors,  so  authors  use  many  different  words  and 
phrases  to  convey  concepts  and  ideas.  This  variability  has  profound  consequences  for  a 
searcher’s  ability  to  retrieve  information. 

All  major  IS&R  systems  do  word  matching  between  the  information  request  and  the  content  of 
documents  as  the  basis  for  the  response  to  an  information  request.  The  cognitive  challenge 
during  search  strategy  development  is  not  simply  to  select  the  words  and  phrases  that  match  your 
interests  and,  at  the  same  time,  reject  the  words  and  phrases  that  do  not  match  your  interests. 
Instead,  the  development  of  an  effective  search  strategy  is  akin  to  peering  into  the  minds  of 
everyone  who  might  have  written  on  the  search  topic  in  order  to  discover  as  many  related  terms 
as  possible. 

At  the  same  time,  IS&R  systems  contain  a  paradox:  words  and  phrases  will  be  associated  with 
information  that  both  matches  and  does  not  match  your  interests.  For  a  simple  and  easy 
example  of  this  paradox,  use  Google  or  another  general  search  engine  to  search  your  own  name. 
Y ou  will  most  likely  retrieve  search  results  related  to  you,  but  it  can  be  something  of  a  shock  to 
see  how  many  other  people  in  the  world  have  the  same  name  as  you  do.  This  simple  search 
demonstrates  a  universal  characteristic  of  IS&R  systems:  a  search  strategy  that  obtains  all  the 
information  of  interest  will  also  retrieve  much  information  that  is  not  of  interest. 

Equally  maddening  for  someone  doing  a  search,  a  retrieval  strategy  designed  to  obtain  only  the 
information  of  interest  will  not  retrieve  all  of  the  infonnation  of  interest.  If  you  perform  the 
search  on  your  given  name,  you  will  not  retrieve  instances  of  your  nickname  unless  you 
specifically  request  it  during  the  search.  But  searching  on  your  nickname  will  also  retrieve  more 
irrelevant  references  connected  to  people  that  share  your  nickname.  Thus,  the  challenge  of  using 
IS&R  systems  is  how  to  balance  the  need  for  comprehensive  retrieval  of  information  with  the 
reality  of  some  wasted  resources  spent  on  infonnation  of  no  interest. 

Fortunately,  there  are  approaches  you  can  use  to  cope  with  the  issues  surrounding  content  and 
the  limitations  of  the  retrieval  capabilities  of  IS&R  systems.  As  part  of  your  search  strategy  you 
can: 

•  Select  words  and  phrases  that  best  represent  the  concepts  and  ideas  of  interest 

•  Use  Boolean  logic  (see  Supplement  1  to  this  section  of  the  FAC  for  a  tutorial)  to 
express  the  relationship  between  the  information  request  and  the  words  in  the 
content  being  searched 

Using  these  approaches  requires  some  creativity  and  experimentation,  which  are  the  cognitive 
challenges  of  infonnation  acquisition.  The  following  hints  provide  a  foundation  for  developing 
search  statements  for  each  of  the  broad  topics  you  identified  in  the  information  plan: 

For  single  words,  the  search  statement  will  consist  of  the  word. 

•  For  more  complex  search  topics  identified  in  the  information  plan,  list  phrases 
related  to  the  topic. 

•  Identity  and  list  synonyms  for  both  single  words  and  phrases 
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•  Remember  to  include  alternate  spelling  of  terms,  plurals,  special  characters,  and 
common  misspellings  as  part  of  your  search  strategy. 

•  Develop  a  series  of  search  statements  that  incorporate  each  of  the  words  and 
phrases  you  identified  as  being  related  to  the  topic. 

•  Incorporate  Boolean  logic  into  the  search  statements  that  include  more  than  one 
word.  As  you  develop  search  statements  for  phrases,  realize  that  an  explicit 
occurrence  of  the  phrase  is  one  search  statement.  You  may  want  to  consider  other 
statements  that  reflect  additional  positional  relationships  of  the  words  in  the  phrase. 
That  is,  perhaps  searching  the  words  in  the  same  sentence,  paragraph,  or  within  so 
many  words  of  one  another  will  also  produce  relevant  results. 


A  NOTE  ABOUT  NON-TEXTUAL  SOURCES  OF  INFORMATION 

As  you  read  our  guidance  about  searching,  remember  that  it  also  applies  to 
searching  for  intelligence  infonnation  that  is  in  a  format  other  than  text,  that  is, 
photographic,  video,  or  other  forms  of  intelligence  information.  While  the  original 
item  you  are  searching  for  is  not  text,  it  most  likely  is  described  and  indexed  using 
text  in  the  IS&R  system.  Our  suggestions  in  this  section  of  the  FAC  apply  to 
searching  the  text-based  description  of  this  material.  If  the  IS&R  has  other  special 
features  designed  to  help  you  search  this  material,  you  should  incorporate  these 
features  in  your  search  as  needed. 


While  creating  an  extensive  list  of  search  terms  is  desirable,  completeness  is  not  usually 
achievable  in  the  early  stages  of  a  search.  Searching  is  a  learning  process  as  well  as  a  retrieval 
process  -  you  will  often  leam  about  new  terms  and  phrases  as  you  search.  These  new  terms  and 
phrases  can  be  searched  on  their  own  during  later  iterations  of  the  search. 

In  the  information  plan  you  identified: 

•  the  information  sources  that  are  likely  to  provide  the  type  of  information  needed  to 
address  the  intelligence  issues 

•  the  IS&R  systems  that  contains  these  information  sources. 

In  all  probability  the  search  and  retrieval  capabilities  offered  by  these  systems  are  not  uniform. 
Some  will  not  provide  for  the  use  of  all  the  Boolean  operators.  Others  will  have  limited  ability  to 
search  within  the  same  sentence  or  paragraph,  for  example.  For  an  example  of  these  variations, 
see  Table  1  for  a  comparison  of  three  major  search  engines  used  on  the  internet. 
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Table  1:  A  Comparison  of  IS&R  Search  Capabilities 


Search  Engine 

Advanced/Boolean 

Operators 

Other  Search  Options 

Goosle 

google.com 

Ranks  based  on  popularity  (# 
of  pages  linked  to). 

AND  (default) 

OR  (capitalized) 

-  to  remove  words  or  phrases. 

+  to  include  common  words 

Wildcard  to  replace 
word(s)  (e.g.,  to  *  or  *  *  *) 

No  truncation. 

Quotes  for  phrase. 

Stems  some  words  (+to  turn 
off). 

Bing 

bing.com 

Optimizes  searching  for 
health,  local,  travel,  and 
shopping 

AND  (default) 

OR  (capitalized) 

NOT  (capitalized)  to  exclude 
words. 

No  truncation. 

Quotes  for  phrase. 

For  dates,  type  the  name  of  the 
month  instead  of  the  calendar 
number. 

Yahoo!  search 
search.yahoo.com 

Ranks  based  on  relevancy 
(occurrence  of  terms). 

AND  (default) 

OR  (capitalized) 

-  to  remove  words  or  phrases. 

+  to  include  common  words. 

No  truncation. 

Quotes  for  phrase. 

Limit  by  date,  language, 
domain,  fde  type,  and  country 
in  advanced  search  fields 

Regardless  of  any  variation  in  retrieval  capabilities  you  might  anticipate,  at  this  stage  you  should 
develop  a  unifonn  series  of  search  statements  as  if  all  of  the  IS&R  systems  you  plan  to  use  have 
a  full  retrieval  capability. 


How  can  I  keep  track  of  my  search  terms,  operators,  and  history? 

Throughout  the  FAC,  we  emphasize  how  writing  benefits  your  cognition.  Consequently,  we 
strongly  recommend  that  you  create  a  search  diary  both  while  developing  a  search  strategy  and 
while  executing  your  search.  You  can  use  the  search  diary  to  record  search  terms  and  statements, 
to  capture  additional  ideas  of  search  terms,  and  to  document  the  IS&R  systems  you  accessed 
during  information  acquisition.  Since  even  simple  searches  can  soon  get  complex,  tracking  all 
search  statements  and  resources  helps  you  make  sure  your  search  is  complete  and  systematic. 

A  search  diary  does  not  have  to  be  complex;  a  simple  table  that  lists  all  of  the  search  terms  and 
any  associated  Boolean  logic  and  where  each  tenn  was  searched  will  be  enough  to  keep  you 
organized  and  thorough.  The  search  diary  can  be  on  paper  or  electronic.  Use  the  format  and 
approach  that  works  best  for  you. 

Another  benefit  of  a  search  diary  is  that  it  can  be  a  helpful  resource  when  you  have  to  determine 
and  convey  information  on  sources,  uncertainties,  and  confidence  levels  for  your  analytic  results. 
The  timelines  you  established  in  the  analysis  plan  may  not  allow  the  execution  of  an  optimum 
search  strategy.  Knowing  what  could  have  been  done  versus  what  was  done  relative  to  the 
information  used  in  the  analysis  can  be  useful  when  you  have  to  communicate  some  measure  of 
certainty  with  your  analytical  results. 
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A  NOTE  ABOUT  SEARCHING  FOR  PERSONALITIES,  FACILITIES,  OR 

EQUIPMENT 

Retrieving  information  about  personalities,  facilities,  or  equipment  is  a  crucial  part  of 
intelligence  analysis  information  acquisition.  The  next  topic  in  this  FAC  section,  “Execution 
of  the  search  strategy”,  assumes  that  you  will  encounter  two  types  of  results  while  searching 
for  personalities,  facilities,  or  equipment: 

•  References  to  personalities,  facilities,  or  equipment  obtained  from  terms  stated 
explicitly  in  the  information  plan 

•  References  that  suggest  other  personalities,  facilities,  and  equipment  terms  that  are 
not  explicitly  listed  in  the  information  plan  and  therefore  are  not  part  of  the  original 
list  of  search  terms 

The  FAC  asks  you  to  treat  the  references  you  retrieve  in  each  of  these  situations  differently. 
The  references  from  the  first  situation  are  evaluated  for  relevance  during  this  stage  of  the 
FAC.  The  references  from  the  second  situation  will  be  evaluated  during  the  “Assignment  of 
Meaning”  phase  of  the  FAC. 


7.3.2.  Execute  the  Search  Strategy 

Once  you’ve  developed  your  initial  search  statements,  you  need  to  execute  them  in  the  IS&R 
systems  that  you  identified  in  the  information  plan.  The  process  of  executing  a  search  also 
includes  examining  the  results  of  each  search  statement  to  determine  whether  the  references 
obtained  answer  the  questions  “Are  these  the  sorts  of  things  I  am  looking  for?  Do  they  pertain  to 
the  intelligence  issue”  In  addition  to  reviewing  the  results  to  see  if  they  are  relevant,  you  can 
modify  the  search  strategy  so  that  you  incorporate  new  terms,  new  resources,  or  to  change  details 
of  the  search. 

What  follows  is  an  idealized  approach  to  the  acquisition  of  relevant  information.  You  would 
modify  this  approach  based  on  the  features  and  limitations  of  each  IS&R  system  you  plan  to  use: 

•  Execute  the  most  restrictive  search  statement  first.  That  is,  search  on  a  single  word, 
an  explicit  phrase,  or  the  personality,  facility,  or  equipment  terms  you  identified  in 
the  information  plan. 

•  Review  the  results  of  the  initial  search  statements  to  identify  and  collect  any 
relevant  results.  Examining  the  title,  table  of  contents,  and  abstracts  or  summaries  is 
a  good  way  to  determine  whether  a  retrieved  item  is  relevant.  Are  the  results  of  the 
search  related  to  the  intelligence  issue? 

•  Screen  the  relevant  information  to  determine  whether  any  alternate  words  or 
phrases  exist  for  the  concepts  and  ideas  of  interest.  If  you  find  new  concepts  or 
terms,  record  them  in  your  search  diary  for  future  search  statement  development. 

•  Use  the  search  diary  to  list  any  terms  related  to  personalities,  equipment,  or 
facilities  associated  with  the  search  topic  but  not  identified  in  the  information 
plan.  Y ou  will  evaluate  and  use  these  further  during  the  assignment  of  meaning 
phase,  which  will  be  discussed  in  a  subsequent  section  of  the  FAC. 
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•  Evaluate  your  search  statements  and  results.  What  was  the  ratio  of  relevant  versus 
non-relevant  information  in  the  search  results?  Is  your  search  strategy  working? 

•  Execute  the  next  least  restrictive  search  statement.  For  example,  if  the  search 
statement  required  that  the  words  in  a  phrase  be  adjacent  to  each  other,  try 
searching  them  with  the  same  sentence  or  within  so  many  words  of  each  other. 

•  Review  and  screen  the  results  you  obtained  with  the  less  restrictive  search 
statement. 

•  Use  the  search  diary  to  record  any  new  terms  related  to  personalities, 
equipment,  or  facilities  associated  with  the  search  topic  but  not  identified  in 
the  information  plan.  You  will  evaluate  and  use  these  terms  further  during  the 
assignment  of  meaning  phase,  which  will  be  discussed  in  a  subsequent  section  of 
the  FAC. 

•  Record  any  new  search  terms  or  phrases. 

•  Compare  the  amount  of  relevant  infonnation  obtained  with  the  current  search 
statement  as  compared  with  the  prior  search  statement.  Consider  the  ratio  of  new 
relevant  information  acquired  and  the  amount  of  non-relevant  infonnation  obtained 
and  make  a  judgment  about  continuing  the  search. 

•  Repeat  the  process  until  all  search  statements  are  executed  or  until  little  or  no 
additional  relevant  information  is  being  acquired. 

•  Repeat  the  process  for  all  of  the  new  words  and  phrases  that  you  recorded  in  the 
search  diary. 

Continue  the  cycle  as  long  as  you  are  retrieving  additional  words  and  phrases,  personalities, 
equipment,  facilities,  or  any  other  relevant  information. 

One  important  reminder  about  information  acquisition:  your  goal  is  to  look  for  and  select  items 
that  are  broadly  relevant.  That  is,  your  thinking  about  whatever  you  retrieve  is  focused  on 
determining  whether  the  results  of  your  search  pertain  to  the  intelligence  issue.  At  this  point,  you 
should  not  attempt  to  form  deeper  judgments  about  the  information’s  meaning  or  value.  These 
judgments  will  be  made  in  later  sections  of  the  FAC. 

Remember  that  the  purpose  of  the  information  acquisition  phase  is  to  obtain  information  which 
bears  upon  the  intelligence  issue  you  are  addressing.  For  this  reason  your  initial  quest  for 
information  will  be,  of  necessity,  the  most  sweeping  retrieval  performed  in  the  course  of  your 
analysis.  If  the  results  you  retrieve  appear  to  be  partially  relevant,  or  if  you  are  in  doubt  whether 
they  are  relevant,  retain  the  results.  This  is  critical  because  if  you  discard  a  document  at  this 
point,  you  have  removed  it  from  consideration  during  later  analysis  activities. 
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A  NOTE  ABOUT  FEW  OR  NO  RESULTS  DURING  INFORMATION 

ACQUISITION 

What  should  an  information  seeker  do  if  a  search  produces  few  or  no  results?  First, 
take  a  look  at  the  basics  of  the  search.  Did  you  spell  the  search  terms  correctly? 

Does  the  topic  area  have  alternate  or  foreign  spelling  that  you  failed  to  use?  Did  you 
select  the  correct  IS&R  systems  for  the  topic  area? 

If  the  basics  of  the  search  were  all  correct,  consider  researching  the  topic  area 
further  in  specialized  thesauri,  manuals,  handbooks,  subject  headings  in  library 
resources,  or  other  indexing  tools  to  better  understand  the  language  associated  with 
the  topic.  Repeat  the  search  with  any  new  terms  you  discover. 

It  is  also  possible  that  the  nonexistence  of  references  is  real.  Lack  of  results  during  a 
search  can  be  informative:  it  may  mean  that  a  certain  area  of  knowledge  or  the 
terminology  associated  with  it  is  not  well-developed.  You  may  have  to  use  search 
terms  from  a  more  general  subject  area  in  order  to  obtain  relevant  results. 

Obtaining  few  or  no  results  is  an  indicator  to  you  that  you  need  to  seek  assistance, 
re-think  your  search  terms  and  search  strategy,  and  try  again. 


7.3.3.  Determination  that  the  Search  is  Complete 

It  can  be  difficult  to  know  when  to  stop  when  you  are  creating  and  executing  a  search.  Both 
internal  and  external  factors  can  contribute  to  your  decision  that  “enough  is  enough”  while 
searching.  Internal  factors  can  include  a  feeling  that  you’ve  retrieved  enough  information  to 
begin  the  next  phase  of  analysis  or  an  indication  that  the  search  terms  or  resource  that  you’re 
using  are  a  dead  end.  Frustration  or  disgust  with  some  aspect  of  the  IS&R  system  you  are  using 
may  make  you  want  to  cease  searching,  but  time  permitting  you  should  persevere  if  more 

information  is  needed.  External  factors  can  include  time  constraints  or  other  constraints  imposed 

88 

by  your  customer. 

No  matter  the  reason,  at  some  point  you  will  conclude  the  information  acquisition  activities 
described  in  this  section  of  the  FAC.  You  may  find  it  helpful  to  record  your  reasons  for  stopping 
a  search  on  a  topic  or  in  a  particular  IS&R  system  for  later  reference. 

7.3.4.  Acquiring  and  Managing  Relevant  Materials  Identified  During  Searching 

As  you  conduct  your  search,  you  will  identify  relevant  references  and  materials.  You  will  need  to 
develop  an  approach  for  organizing  the  materials  you  retrieve.  Some  analysts  prefer  to  have  their 
search  results  stored  in  digital  format;  other  analysts  prefer  hard  copy.  As  with  other  aspects  of 
the  FAC,  we  only  suggest  that  an  organizing  approach  is  necessary  -  each  analyst  needs  to 
develop  a  system  for  storing  and  handling  retrieved  results  that  works  for  him  or  her. 


Prabha,  C.,  Connaway,  L.  S.,  Olszewski,  L.,  Jenkins,  L.  R.  “What  is  Enough?  Satisficing  Information  Needs.” 
Journal  of  Documentation,  Vol.  63,  No.  1,  2007,  pp.  74-89. 
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An  organizing  approach  can  help  you  address  another  common  infonnation  acquisition  issue 
discussed  in  the  infonnation  plan  section.  If  you  use  multiple  IS&R  systems,  you  are  highly 
likely  to  obtain  the  same  information  multiple  times.  Some  of  the  results  you  obtain  will  be 
duplicates  of  the  same  information,  repeated  in  different  IS&R  systems.  This  is  a  fairly  common 
situation  and  one  that  you  probably  anticipate  any  time  you  use  an  IS&R  system. 

However,  a  different  type  of  duplication  may  also  exist  in  the  results  obtained  with  an  IS&R 
system.  That  is,  you  may  have  information  from  the  same  source  that  is  duplicated  many  times 
within  the  same  IS&R  system  but  within  different  reports.  For  example,  you  might  retrieve  100 
reports  and  find  that  they  are  from  the  exact  same  original  HUMINT  report  from  the  same 
source.  If  you  are  not  attentive  to  this  form  of  duplication,  you  might  think  that  your  search  had 
retrieved  100  independent  references.  In  this  case,  lots  of  references  are  not  equivalent  to  lots  of 
information  on  a  given  topic. 

A  systematic  approach  to  organizing  the  information  you  retrieve  helps  you  identify  and  manage 
duplicates  and  weed  out  duplicate  instances  from  within  the  same  and  different  IS&R  systems. 
Managing  duplicates  helps  you  develop  a  more  realistic  picture  of  the  results  of  your  search. 

Depending  on  the  IS&R  system,  the  items  you  retrieve  may  only  be  abstracts  or  summaries  of 
the  original  publication,  image,  or  other  type  of  material.  In  order  to  fully  evaluate  the  retrieved 
references  for  relevance,  you  will  have  to  obtain  the  original  item  in  its  entirety.  You  may  find  it 
very  helpful  to  use  your  organization’s  information  providers  as  a  resource  for  locating,  ordering, 
and  managing  the  procurement  of  original  materials. 

7.4  Summary 

Information  acquisition  is  not  just  a  mechanical  approach  to  obtaining  information.  You  prepared 
yourself  for  infonnation  acquisition  by  learning  a  great  deal  about  the  intelligence  issue  and  the 
customer’s  needs  during  the  preceding  FAC  activities.  Bringing  this  mental  state  to  information 
acquisition  allows  you  to  be  more  attentive  while  constructing  a  search  and  evaluating  the 
retrieved  information  for  relevance  to  the  intelligence  issue. 

In  addition,  information  acquisition  can  be  an  opportunity  for  learning  more  about  the 
intelligence  issue  as  you  encounter  new  concepts,  terms,  personalities,  facilities,  and  equipment 
during  searching  and  evaluating.  We  have  emphasized  throughout  the  FAC  that  working  on  an 
intelligence  issue  is  an  iterative  process;  the  knowledge  you  gain  during  information  acquisition 
can  be  a  critical  source  of  information  for  modifying  the  intelligence  issue,  analysis  plan,  or 
information  plan. 

Finally,  dare  we  say  it?  It  can  be  intensely  interesting  to  be  actively  involved  with  looking  for 
information  related  to  the  intelligence  issue  and  evaluating  it  for  relevance.  The  results  you 
obtain  during  infonnation  acquisition  can  stimulate  more  cognitive  engagement  with  the 
intelligence  issue.  The  results  certainly  pave  the  way  for  a  later  FAC  section:  assignment  of 
meaning  to  the  infonnation  you  retrieve. 
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7.5  Supplement  1:  Boolean  Searching,  A  Primer89 

The  contents  of  IS&R  systems  are  usually  searched  according  to  the  rules  of  computer  database 
searching.  The  principles  of  Boolean  logic  fonn  the  basis  for  much  of  the  database  searching  you 
do.  Boolean  logic  is  a  description  of  the  relationships  that  can  exist  among  search  terms.  The 
name  Boolean  comes  from  the  name  of  British-born  Irish  mathematician  George  Boole. 

Boolean  logic  consists  of  three  logical  operators: 

•  OR 

•  AND 

•  NOT 

Each  logical  operator  can  be  visually  described  by  using  Venn  diagrams.  Venn  diagrams  were 
conceived  around  1880  by  John  Venn  to  represent  relationships.  They  commonly  consist  of 
overlapping  circles  with  areas  highlighted  in  different  ways  depending  on  the  logical  operator.  In 
this  section  we  use  Venn  diagrams  to  show  the  relationships  between  search  terms  when  various 
logical  operators  are  used. 

7.5.1.  OR  Logic 

The  first  operator  we’ll  discuss  is  OR. 


Figure  1:  OR  Logic 

Equivalent  Search  Statement:  college  OR  university 

Question:  I  would  like  information  about  college. 

In  this  search,  we  will  retrieve  records  in  which  AT  LEAST  ONE  of  the  search  terms  is  present. 

We  are  searching  on  the  terms  college  and  also  university  since  documents  containing  either  of 

these  words  might  be  relevant. 

This  is  illustrated  by: 

•  the  shaded  circle  with  the  word  college,  which  represents  all  of  the  records  that 
contain  the  word  "college" 

•  the  shaded  circle  with  the  word  university,  which  represents  all  of  the  records  that 
contain  the  word  "university" 

•  the  shaded  overlap  area,  which  represents  all  of  the  records  that  contain  both 
"college"  and  "university" 


89  Adapted  from  http://www.internettutorials.net/boolean.asp  as  viewed  on  15  January  2010. 
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OR  logic  is  most  commonly  used  to  search  for  synonymous  terms  or  concepts. 
Here  is  an  example,  using  search  results,  of  how  OR  logic  works: 


Search  Terms 

Results 

College 

396,482 

University 

590,791 

College  OR  University 

819,214 

OR  logic  collates  the  results  to  retrieve  all  the  unique  records  containing  one  term,  the  other 
term,  or  both  terms. 

As  a  general  rule,  the  more  terms  or  concepts  combined  with  OR  logic  in  a  search  statement,  the 
more  results  retrieved. 


Figure  2:  OR  Logic  with  Multiple  Terms 

Equivalent  Search  Statement:  college  OR  university  OR  campus 

The  search  results  for  this  example  are: 


Search  Terms 

Results 

College 

396,482 

University 

590,791 

College  OR  University 

819,214 

College  OR  University  OR  Campus  929,677 
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7.5.2. 


AND  Logic 


Figure  3:  AND  Logic 

Equivalent  Search  Statement:  poverty  AND  crime 

Question:  I'm  interested  in  the  relationship  between  poverty  and  crime. 

•  In  this  search,  we  retrieve  records  in  which  BOTH  of  the  search  terms  are  present. 

•  This  is  illustrated  by  the  shaded  area  overlapping  the  two  circles  representing  all  the 
records  that  contain  both  the  word  "poverty"  and  the  word  "crime." 

•  Notice  that  the  search  does  not  retrieve  any  records  with  only  "poverty"  or  only 
"crime." 

Here  is  an  example,  using  search  results,  of  how  AND  logic  works: 


Search  Terms 

Results 

poverty 

76,342 

crime 

348,252 

poverty  AND  crime  12,998 

As  a  general  rule,  the  more  terms  or  concepts  combined  in  a  search  with  AND  logic,  the  fewer 
results  retrieved. 


Figure  4:  AND  Logic  with  Multiple  Terms 

Equivalent  Search  Statement:  poverty  AND  crime  AND  gender 


123 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 
88ABW/PA  cleared  15  Aug  12,  88ABW-20 12-4458 


The  search  results  for  this  example  are: 


Search  Terms 

Results 

Poverty 

76,342 

Crime 

348,252 

Poverty  AND  Crime 

12,998 

Poverty  AND  Crime  AND  Gender 

1,220 

7.5.3.  NOT  Logic 


Figure  5:  NOT  Logic 

Equivalent  Search  Statement:  cats  NOT  dogs 

Question:  I  want  information  about  cats,  but  I  don't  want  to  see  anything  about  dogs. 

•  In  this  search,  we  retrieve  records  in  which  ONLY  ONE  of  the  terms  is  present,  the 
one  we  have  selected  by  our  search. 

•  This  is  illustrated  by  the  shaded  area  with  the  word  cats  representing  all  the  records 
containing  the  word  “cats." 

•  No  records  are  retrieved  in  the  area  overlapping  the  two  circles  where  the  word 
"dogs"  appears,  even  if  the  word  "cats"  appears  there  too. 

Here  is  an  example,  using  search  results,  of  how  NOT  logic  works: 


Search  Terms 

Results 

Cats 

86,747 

Dogs 

130,424 

Cats  NOT  Dogs 

65,223 

NOT  logic  excludes  records  from  your  search  results.  Be  careful  when  you  use  NOT :  the  term 
you  do  want  may  be  present  in  an  important  way  in  documents  that  also  contain  the  word  you 
wish  to  avoid.  For  example,  consider  a  Web  page  that  includes  the  statement  that  "cats  are 
smarter  than  dogs."  The  search  illustrated  above  would  exclude  this  document  from  your  results. 

7.5.4.  Combined  AND  and  OR  Logic 

Question:  I  want  information  about  the  behavior  of  cats. 

Equivalent  Search  Statement:  behavior  AND  (cats  OR  felines) 

You  can  combine  both  AND  and  OR  logic  in  a  single  search,  as  shown  in  the  search  statement 
above. 
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The  use  of  parentheses  in  this  search  is  known  as  forcing  the  order  of  processing.  In  this  case,  we 
surround  the  OR  words  with  parentheses  so  that  the  search  engine  processes  the  two  related 
terms  (in  this  example,  “cats”  and  “felines”)  as  a  unit.  The  search  engine  uses  AND  logic  to 
combine  this  result  with  the  second  concept  (in  this  example,  “behavior”).  Using  this  method,  we 
are  assured  that  the  semantically-related  OR  terms  are  kept  together  as  a  logical  unit. 

7.5.5.  Restrictive  AND  Variations 

AND  is  a  useful  Boolean  operator,  but  it  can  be  too  broad  when  you  are  trying  to  retrieve 
information  on  a  specific  topic.  The  two  words  connected  by  AND  may  be  in  the  same 
document,  but  they  may  refer  to  completely  different  concepts  and  have  no  relationship  to  each 
other.  To  combat  this  issue,  many  IS&R  systems  have  implemented  what  are  known  as 
restrictive  AND  or  proximity  operators  for  use  during  a  search.  Restrictive  AND  operators  allow 
you  to  specify  relationships  between  terms  that  are  more  precise  than  the  fact  that  the  two  terms 
are  in  the  same  document. 

For  example,  some  IS&R  systems  use  an  operator  known  as  NEAR.  The  characteristics  of  the 
IS&R  system  determine  exactly  what  NEAR  represents,  but  it  usually  means  that  the  two  words 
in  the  search  statement  are  within  a  few  words  of  each  other  in  the  document  retrieved.  Other 
restrictive  AND  operators  can  limit  the  search  terms  to  the  same  paragraph  or  sentence,  or 
require  the  terms  to  be  adjacent  to  each  other. 


Table  2  shows  how  various  restrictive  AND  operators  affect  the  results  of  a  sample  search  for 
‘information  storage  and  retrieval’  in  an  IS&R  system. 

Table  2:  Effects  of  Restrictive  AND  on  Search  Results 


Operator 

Documents  Retrieved 

Term  Occurrences 

Exact  phrase 

57 

75 

Within  5  words 

72 

136 

Within  10  words 

83 

264 

Within  25  words 

94 

709 

AND  (same  document) 

225 

252,782 

As  a  general  rule,  the  use  of  restrictive  AND  operators  improves  the  relevance  of  your  search.  If 
you  were  searching  the  sample  topic  used  for  the  chart,  you  would  want  to  evaluate  the  results  of 
each  search  statement  to  determine  the  best  restrictive  AND  operator  for  your  topic. 


In  order  to  effectively  use  the  IS&R  systems  available  to  you  during  intelligence  analysis,  you 
need  to  familiarize  yourself  with  the  restrictive  AND  or  proximity  operators  available  in  each 
system  you  plan  to  use.  We  encourage  you  to  experiment  with  restrictive  AND  operators  and 
inspect  the  results  of  searches  with  each  operator  to  judge  the  relevance  of  what  you  retrieve. 
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8.0 

8.1 


ANALYSIS  TECHNIQUES 
Now  What? 


At  this  stage  of  the  FAC,  you  have  explored  and  developed  your  intelligence  issue,  established  a 
plan  for  addressing  the  issue,  developed  a  plan  to  acquire  infonnation  relevant  to  the  issue, 
acquired  information  based  on  that  plan,  and  selected  information  relevant  to  the  issue.  Now  you 
have  a  collection  of  information  that  you  will  use  to  develop  an  assessment  that  addresses  the 
intelligence  issue. 

The  prime  question  now  is  “from  this  collection  of  information,  how  do  I  create  an  assessment?” 
Thoughtful  reading  of  the  information  is  essential,  but  will  not  be  enough.  Rarely  will  the 
appropriate  assessment  be  self  evident  in  the  infonnation.  The  basis  for  developing  your 
assessment  will  be  there  but  it  will  be  implicit,  hidden,  buried,  and  fragmented.  From  this 
collection  of  information  and  other  information  that  you  will  acquire  as  you  proceed,  you  must 
determine  the  meaning  of  both  the  individual  pieces  of  information  and  the  meaning  of  the 
aggregate  information  and  meanings  to  create  an  explicit,  credible,  and  coherent  assessment. 

Determining  the  meaning  of  information  to  create  an  assessment  is  the  essence  of  analysis.  We’ll 
provide  information  in  this  FAC  section  to  assist  you  with  developing  and  focusing  your  mental 
framework  in  preparation  for  this  crucial  aspect.  We  won’t  ask  you  to  prepare  any  written 
materials  with  this  section.  But,  we’ll  encourage  you  to  mentally  engage  with  the  topics 
described  that  affect  your  thinking  and  reflect  upon  those  topics,  considering  your  own  behaviors 
and  capabilities.  As  you’ll  see,  a  recurring  theme  about  thinking  is  the  importance  of  motivation 
and  engagement  relative  to  the  topic. 

In  this  section,  you’ll  see  some  familiar  themes  from  the  prior  FAC  sections,  “Analysts  as 
Individuals”  and  “Thinking  about  Thinking.”  We’ll  elaborate  upon  many  of  the  concepts 
introduced  in  that  section.  One  of  these  is  “thinking  dispositions.”  The  nature  of  your  thinking 
dispositions  is  closely  related  to  your  motivation  and  engagement.  It  is  essential  for  you  to  be 
keenly  aware  of  these  concepts  about  thinking  as  you  embark  upon  the  Assignment  of  Meaning 
described  in  the  next  section. 

As  we  stated  in  the  “Introduction  to  the  FAC”  section,  Kent  identifies  two  steps  after  all  the 
information  has  been  acquired: 

•  Critical  evaluation  of  the  data  thus  assembled.  (Step  4) 

•  Study  of  the  evaluated  data  with  the  intent  of  finding  some  inherent  meaning.  (Step 
5)90 

Evaluating  information  and  detennining  the  meaning  of  information  has  always  been  an  integral 
part  of  the  practice  of  intelligence  analysis.  However,  as  the  volume  of  information  has 
increased,  the  issues  have  become  more  complex  and  the  meanings  more  subtle.  Detennining 
meaning  has  rapidly  become  more  burdensome  and  cognitively  difficult.  Consequently,  there  has 
been  an  increasing  amount  of  attention  in  the  intelligence  literature  on  understanding  traditional 
analysis  methods  and  techniques,  as  well  as  the  development  of  new  methods  and  techniques  to 
supplement  or  replace  what  has  been  used. 


90  Kent,  S.  (1949).  Strategic  Intelligence  for  American  World  Policy,  Princeton  University  Press,  Princeton,  NJ, 
1949,  pp.  157-158. 
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There  are  many  traditional  and  new  analysis  techniques  for  detennining  the  meaning  of 
information.  In  the  “Analysts  as  Individuals”  section,  we  emphasized  that  to  improve  your 
judgments  about  the  meaning  of  information  you  must  develop  a  realistic  appraisal  of  your 
cognitive  abilities  and  the  content  of  your  mind.  As  stated  previously,  the  infonnation  provided 
in  this  section  is  to  aid  you  with  this. 

Because  the  intelligence  literature  focuses  on  new  techniques,  we’ll  also  provide  you  information 
about  some  of  those  techniques.  These  techniques  are  often  referred  to  as  structured  analytic 
methods  or  techniques.  In  this  FAC  section  we’ll  refer  to  those  approaches  and  techniques 
identified  in  the  intelligence  literature  as  SATs.  We’ll  also  provide  information  for  you  to 
consider  in  choosing  appropriate  analysis  approaches  and  techniques  to  aid  you  in  evaluating  and 
assigning  meaning  to  infonnation,  both  individual  pieces  of  information  and  the  aggregation  of 
the  infonnation.  As  we’ll  explain  later,  with  few  exceptions,  the  SATs  identified  in  the 
intelligence  literature  are  not  applicable  to  assigning  meaning  to  individual  pieces  of 
information.  They  are  largely  focused  on  dealing  with  the  aggregation  of  information. 

To  begin  our  discussion,  we’ll  start  with: 

•  The  role  of  SATs  as  described  in  the  intelligence  literature 
This  discussion  will  be  followed  by  information  on: 

•  The  cognition  of  analysis,  including  the  functions  and  types  of  thinking  that  occur 
in  the  cognitive  domain,  as  well  as  cognitive  factors  that  impact  the  quality  of 
thinking  during  analysis 

•  SATs  and  the  cognition  of  analysis 

In  subsequent  sections  of  the  FAC,  we’ll  introduce  additional  analytic  approaches  that  we 
recommend  you  use  to  organize  your  information  and  assign  meaning  to  it,  marshal  your 
information,  and  formulate  and  evaluate  hypotheses. 

8.2  SATs 

There  are  various  definitions  for  what  constitutes  a  SAT  and  much  debate  about  the  value  of 
using  them,  both  for  individual  analysts  and  for  analytic  groups.  While  the  debate  continues, 
there  has  been  little  systematic  research  on  the  application  of  SATs  to  intelligence  analysis. 
Opinions  vary,  but  many  agree  that  use  is  not  widespread  among  analysts. 

Even  the  proponents  of  SATs  acknowledge  that  the  most  frequently  used  way  of  doing  analysis 
is: 

. .  .the  intuitive  method  (also  known  as  "read  a  bunch  of  stuff,  think  about  it  for  a  bit  and 
then  write  something")  remains  the  most  popular  method  for  producing  intelligence 
analysis...91 

. .  .the  traditional  CIA  method  of  analysis . Read  as  much  as  you  have  time  to  read  that 

day;  Think  about  it  and  suck  an  answer  out  your  thumb;  Write  it  down  in  as  crisp  a 
manner  as  possible.92 


91  Wheaton,  K.  Top  5  Intelligence  Analysis  Methods  (list).  2008.  Available: 
http://sourcesandmethods.blogspot.com/2008/12/top-5-intelligence-analysis-methods.html,  pp.  1. 

92Marrin,  S.  "Intelligence  Analysis:  Structured  Methods  or  Intuition?"  American  Intelligence  Journal,  Vol.  25,  No. 
1,  2007,  p.  9. 
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Traditional  intelligence  assessment  methodology  has  always  been 
historiographical93 . . .  strictly  descriptive ... 94 

Because  of  perceived  shortcomings  in  the  results  of  intuitive  analysis,  the  amount  of  writing  and 
development  focused  on  SATs  has  grown.  How  are  such  techniques  defined?  One  view  is  that 
“they  are  those  techniques  which  have  a  fonnal  or  structured  methodology  that  is  visible  to 
external  observers.”95  A  different  view  is  that  to  be  useful  for  professional  purposes,  methods 
“are  processes  that  produce  or  substantially  help  the  analyst  produce  estimative  results”. 
According  to  this  view,  often  “method”  is  used  informally  by  analysts  to  include  ways  to 
improve  thinking.96  Others  see  SATs  and  methodology  use  as  the  application  of  the  scientific 
method  to  intelligence  analysis. 

According  to  the  proponents,  the  benefits  associated  with  the  use  of  SATs  are: 

SATs  are  the  enablers  of  collaboration.  Thev  are  the  process  bv  which  effective 

no 

collaboration  occurs. 

The  primary  value  of  analytic  techniques  or  structured  methods  is  that  they  provide  a  way 
to  account  for  the  analytic  judgment;  an  analytic  "audit  trail"  as  it  were.99 

From  this  diversity  of  opinions,  you  can  correctly  infer  that  some  SATs  are  more  suitable  for 
certain  types  of  analysis  or  are  more  pertinent  depending  on  the  issues  being  addressed.  As  an 
analyst,  you  are  the  intended  beneficiary  of  these  many  SATs  and  can  select  those  that  will 
benefit  your  analysis  from  all  of  those  available. 

What  should  you  consider  when  trying  to  decide  if  you  should  use  a  particular  SAT?  First,  an 
understanding  of  the  characteristics  of  the  intelligence  issue  is  essential.  Most  intelligence  issues 
relate  to  a  future  state  of  affairs;  but  is  the  future  defined  as  tomorrow,  next  month,  next  year,  or 
next  decade?  Also,  some  intelligence  issues  deal  with  the  actions  and  decisions  of  individuals, 
groups,  nations  or  international  entities,  and  these  characteristics  may  affect  your  choice  of 
SATs.  While  understanding  the  nature  of  the  issue  is  necessary,  it  is  not  enough  to  make  an 
appropriate  choice  about  the  SATs  to  use.  To  select  appropriate  SATs,  you  should  also  consider 
whether  the  SAT  helps  you  assign  meaning  to  individual  pieces  of  information,  the  aggregation 
of  information,  or  both.  We  will  return  to  this  important  topic  at  the  end  of  this  FAC  section.  In 
addition,  it  is  essential  to  consider  the  nature  of  the  cognition  that  you  are  trying  to  aid  by  using 
the  SAT.  This  requires  an  understanding  of  the  cognition  associated  with  analysis,  as  well  as 
what  aspects  of  cognition  are  aided  by  specific  SATs. 

The  following  discussion  elaborates  on  the  infonnation  provided  in  the  “Thinking  about 
Thinking”  section  of  the  FAC  and  discusses  the  cognition  of  analysis  to  (1)  help  prepare  you  for 


93  Historiography  is  the  history  of  history,  the  aspect  of  history  and  of  semiotics  that  considers  how  knowledge  of 
the  past,  either  recent  or  distant,  is  obtained  and  transmitted. 

94Smith,  T.J.  "Predictive  Warning:  Teams,  Networks,  and  Scientific  Method,"  in  Analyzing  Intelligence:  Origins, 
Obstacles,  and  Innovations,  eds.  R.Z.  George  &  J.B.  Bruce,  Georgetown  University  Press,  Washington,  D.C.,  2008, 
p.  269. 

95Marrin  p.  7. 

96Wheaton  p.  1-2. 

9jSmith  p.  268. 

98Heuer,  R.J.  The  Evolution  of  Structured  Analytic  Techniques.  Speech  presented  to  the  National  Academy  of 
Science,  Washington,  D.C,  December  8,  2009,  p.  3. 

"Marrin  p.  7. 
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the  assignment  of  meaning  that  will  occur  in  the  next  FAC  section,  and  (2)  provide  information 
for  you  consider  in  choosing  SATs  to  use  in  your  analysis. 

8.3  Cognition  of  Analysis 

As  we  introduced  in  the  FAC  sections  “Analysts  as  Individuals”  and  “Thinking  about  Thinking,” 
a  very  large  body  of  literature  exists  on  the  topic  of  thinking.  Since  man  created  written  records, 
philosophers  and  scientists  have  struggled  with  the  concept  of  what  thinking  is  and  how  thinking 
is  influenced  by  many  factors.  For  our  purposes  the  following  definition  of  thinking  relates  to 
doing  intelligence  analysis: 

Thinking,  in  its  broadest  sense,  is  the  search  for  meaning.  It  consists  either  of  finding 
meaning  assumed  to  exist  already  or  of  making  meaning  out  of  something  that  has  no 
readily  apparent  meaning.  It  is,  as  John  Dewey  wrote  years  ago,  "that  operation  in  which 
present  facts  suggest  other  facts  (or  truths)  in  such  a  way  as  to  induce  belief  in  the  latter 
upon  the  ground  or  warrant  of  the  former."  Thinking,  in  short,  is  the  mental  process  by 
which  individuals  make  sense  out  of  experience.100 

To  provide  more  understanding  about  thinking,  next  we’ll  discuss  the  cognitive  domain 
functions,  types  of  thinking  in  the  cognitive  domain,  and  cognitive  factors  that  affect  thinking 
quality. 

8.3.1.  Cognitive  Domain  Functions 

While  much  disagreement  continues,  general  agreement  exists  that  the  cognitive  domain  as 
applied  to  decision  making,  problem  solving  or  assigning  meaning  consists  of  knowledge, 
comprehension,  application,  analysis,  synthesis,  and  evaluation.101  However,  this  description  of 
the  cognitive  domain  does  not  discuss  the  functions  that  provide  input  to  the  cognition,  namely 
reading  and  learning.  These  are  the  precursor  functions  used  to  establish  the  knowledge  used  in 
the  cognition. 

If  we  list  the  cognitive  domain  functions  (i.e.,  reading,  learning,  knowledge,  comprehension, 
application,  analysis,  synthesis,  and  evaluation),  it  conveys  the  impression  of  a  linear  step-by- 
step  process.  In  practice  that  is  not  the  way  we  generally  think. 

. .  .although  we  can  isolate  and  hierarchically  order  separate  levels  of  cognitive  processing 

that  are  active  in  reading,  such  as  recognition,  recall,  comprehension,  application, 

analysis,  synthesis,  and  evaluation  (RAND,  2002),  the  reader  certainly  does  not 

experience  them  as  separate  and  sequenced.  Rather,  they  are  more  likely  to  occur 

1 02 

simultaneously,  in  an  overlapping  and  interactive  fashion. 


100Beyer,  B.K.  Practical  Strategies  for  the  Teaching  of  Thinking,  Allyn  and  Bacon,  Boston,  MA,  1987,  p.  16. 

NOTE:  The  terms  “fact”  as  used  by  Dewey  and  “data”  as  used  by  Kent  have  different  meanings  than  the 
contemporary  meaning  assigned  to  these  words.  Their  intent  can  now  be  best  expressed  by  substituting  the  word 
“information.” 

'"’Alexander,  P.A.,  Dinsmore,  D.L.,  Fox,  E.,  Grossnickle,  E.M.,  Loughlin,  S.M.,  Maggioni,  L.,  Parkinson,  M.M.  & 
Winters,  F.I.  Higher-Order  Thinking  and  Knowledge:  Domain-General  and  Domain-Specific  Trends  and  Future 
Directions.  Undated  Manuscript,  pp.  4. 

102Fox,  E.  &  Alexander,  P.A.  "Text  Comprehension:  A  Retrospective,  Perspective,  and  Prospective,"  in  Handbook  of 
Research  on  Reading  Comprehension,  eds.  S.E.  Israel  &  G.G.  Duffy,  Routledge,  New  York,  NY,  2009,  p  234. 
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In  addition  to  being  overlapping  and  interactive,  each  function  is  dependent  on  the  results  of  the 

i  ni 

antecedent  functions.  This  is  an  extremely  important  finding  that  has  significant  ramifications 
for  the  assignment  of  meaning  that  we  will  discuss  in  subsequent  FAC  sections. 

As  mentioned  earlier,  little  agreement  exists  on  how  thinking  is  done,  but  there  is  growing 
agreement  on  the  attributes  of  the  thinker  and  how  these  relate  to  the  quality  of  what  is  done.  An 
important  question  to  consider  is  how  the  SATs  relate  to  improving  these  attributes.  To  better 
understand  the  cognitive  attributes  associated  with  quality  thinking  and  the  potential  utility  of 
various  SATs,  let’s  take  a  closer  look  at  the  “input  functions”  of  reading  and  learning. 

Reading:  Reading  competence  has  received  considerable  attention  for  the  last  30-40  years.  Most 
of  this  attention  has  been  focused  on  teaching  reading  to  children  and  youths.  This  dominant 
focus  on  children  may  imply  that  the  majority  of  adults  can  read  competently.  Unfortunately, 
other  research  results  do  not  support  this  assumption.  Differences  in  reading  competence  have 
been  measured  in  college  students.104  Others  have  written  on  changes  in  reading  competence 
across  one’s  lifespan,105  as  well  as  the  importance  of  knowledge,  interests,  and  strategies  to 
achieve  expert-level  reading  competence.106  Thus,  one  cannot  assume  that  that  all  analysts  can 
and  do  read  with  high  levels  of  comprehension.  Since  the  overwhelming  majority  of  information 
used  in  analysis  requires  reading,  the  potential  impact  on  subsequent  analysis  is  consequential. 

To  help  you  reflect  upon  your  own  cognition  and  behavior,  we  suggest  that  you  consider  the 
following  descriptions  relative  to  your  reading. 

So  what  is  a  competent  reader? 

. .  .the  proficient  adult  reader  can  read  a  variety  of  materials  with  ease  and  interest,  can 
read  for  varying  purposes,  and  can  read  with  comprehension  even  when  the  material  is 
neither  easy  to  understand  nor  intrinsically  interesting.107 

What  are  the  attributes  of  a  competent  reader?  Generally  three  areas  have  been  identified  that  are 
instrumental  in  reading  competence.108 

•  Linguistic  and  Metalinguistic  Knowledge  -  “Because  reading  requires  access  to  and 
interpretation  of  written  language,  competent  readers  must  be  able  to  "break  the 
code”...  entails  the  individual’s  cognizance  of  language  concepts  and 
conventions... linguistic  and  metalinguistic  knowledge  must  be  sufficiently 
developed  in  readers  to  pennit  them  to  decode  and  encode  print  rapidly  and 
accurately...”  [emphasis  added] 


Beyer,  B.K.p.  30. 

104Fox,  E.,  Dinsmore,  D.L.,  Maggioni,  L.  &  Alexander,  P.A.  "Factors  Associated  with  Undergraduates'  Success  in 
Reading  and  Learning  from  Course  Texts",  2009  Annual  Meeting  of  the  American  Educational  Research 
Association,  San  Diego,  CA,  April  13-17,  2009. 

105 Alexander,  P.A.  The  Path  to  Competence:  A  Lifespan  Developmental  Perspective  on  Reading  2006.  Manuscript. 
106Fox,  E.,  Maggioni,  L.,  Dinsmore,  D.L.  &  Alexander,  P.A.  "The  Multi-Layered  Reading  Goals  of  Expert  Readers: 
Bridging  between  Knowledge,  Interest,  and  Strategy  use",  2008  Annual  Meeting  of  the  American  Educational 
Research  Association,  New  York,  NY,  March  24-28,  2008. 

107Snow,  C.E.  Reading  for  Understanding:  Toward  a  Research  and  Development  Program  in  Reading 
Comprehension,  RAND  Corporation,  Santa  Monica,  CA,  2002,  p.  xiii. 

“Alexander,  P.A.,  Garner,  R.,  Sperl,  C.T.  &  Flare,  V.C.  "Fostering  Reading  Competence  in  Students  with  Learning 
Disabilities,"  in  Learning  about  Learning  Disabilities,  ed.  B.Y.L.  Wong,  2nd  edn,  Academic  Press,  San  Diego,  CA, 
1998,  pp.  343-349. 
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•  Conceptual  Understanding  -  .  .the  meaning  that  students  construct  from  text  is 

directly  related  to  the  framework  of  conceptual  knowledge  that  they  bring  to  that 
text.”  [emphasis  added] 

•  Strategic  Ability  -  “To  achieve  competence  in  reading,  learners  must  function  with 
an  awareness  of  their  behavior,  monitor  the  cognitive  aspects  of  their 
perfonnance,  establish  their  goals  for  learning,  as  well  as  regulate  their  efforts 
towards  those  goals. ..competent  readers  must  be  active,  strategic  text 
processors...”  [emphasis  added] 

•  Motivation  for  Learning  -  “. .  .competence  in  reading... is  as  much  an  investment  of 
the  spirit  as  it  is  the  mind. . .”  Requiring  “. .  .a  suitable  goal,..,  an  interest  in  the 
topic  or  task  at  hand. . .,  a  belief  in  themselves. . .,  a  sense  of  self-control  or  self- 
determination. . .”  [emphasis  added] 

Some  of  the  attributes  that  affect  reading  competence  are  the  same  ones  that  positively  affect 
other  functions  in  the  cognitive  domain,  such  as  knowledge,  awareness  of  behavior,  goals,  and 
motivation.  Our  review  indicates  that  the  SATs  do  little  to  directly  improve  reading  ability.  But, 
they  may  help  you  generally  improve  your  knowledge,  the  content  of  your  mind,  which  should 
positively  affect  subsequent  reading. 

Learning:  Reading  has  received  lots  of  attention,  but  it  pales  in  comparison  to  the  attention  on 
learning.  Many  facets  of  what  determines  the  ability  to  learn  have  been  investigated.  Some  of 
these  are  the  same  as  for  reading,  namely  knowledge  and  strategic  abilities/processes.  Many  of 
the  others  are  familiar  and  have  received  lots  of  attention  in  the  popular  press,  such  as  teacher 
quality,  classroom  size,  and  teacher  to  student  ratios.  While  it  hasn’t  received  much  publicity, 
beliefs  about  the  nature  of  knowledge,  i.e.,  epistemological  beliefs,  play  a  critical  role  in 
learning.109  110 

What  are  epistemological  beliefs?  This  is  one  description: 

...CERTAINTY  of  knowledge,  ranging  from  knowledge  is  absolute  to  knowledge  is 
tentative. . . 

...STRUCTURE  of  knowledge,  ranging  from  knowledge  is  organized  as  isolated  bits  and 
pieces  to  knowledge  are  organized  as  highly  interwoven  concepts... 


. . .  SOURCE  of  knowledge,  ranging  from  knowledge  is  handed  down  by  authority  to 
knowledge  is  derived  through  reason. . . 

. .  .CONTROL  of  knowledge  acquisition,  ranging  from  the  ability  to  learn  is  fixed  at  birth 
to  the  ability  to  learn  can  be  changed. . . 


109Schommer,  M.  &  Walker,  K.  "Are  Epistemological  Beliefs  Similar  Across  Domains?"  Journal  of  Educational 
Psychology’ ,  Vol.  87,  No.  3,  1995,  pp.  424-432. 

ll0Harteis,  C.,  Gruber,  H.  &  Lehner,  F.  "Epistemological  Beliefs  and  their  Impact  on  Work,  Subjectivity,  and 
Learning,"  in  Work,  Subjectivity  and  Learning:  Understanding  Learning  through  Working  Life,  eds.  S.  Billett,  T.J. 
Fenwick  &  M.  Somerville,  Springer,  Dordrecht,  The  Netherlands,  2006,  pp.  123-140. 
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. . .  SPEED  of  the  knowledge  acquisition,  ranging  from  knowledge  is  acquired  quickly  to 
not-at-all  to  knowledge  is  acquired  gradually. 111 

Epistemological  beliefs  at  the  lower  end  of  the  continuum  (e.g.,  knowledge  is  absolute, 
organized  in  isolated  bits  and  pieces,  handed  down  by  authority,  with  the  ability  to  leam  fixed  at 
birth)  are  characterized  as  naive  beliefs.  Those  beliefs  at  the  upper  end  of  the  continuum  (e.g., 
knowledge  is  tentative,  organized  as  interwoven  concepts,  derived  through  reason,  with  an 
ability  to  leam  that  can  change)  are  characterized  as  sophisticated  beliefs.  In  general,  beliefs 

1 13 

become  more  sophisticated  with  age  and  education. 

The  fact  that  epistemological  beliefs  can  have  such  a  significant  effect  on  one’s  learning 
capacity,  in  contrast  with  other  things  normally  associated  with  learning  such  as  intelligence  or 
memory,  may  seem  surprising.  But,  perhaps  it  shouldn’t  be  since  we  discussed  in  Analysts  As 
Individuals,  Knowing  Yourself,  that  one’s  ability  is  more  affected  by  praise  of  effort  if  you 
believe  that  intelligence  is  malleable  than  if  you  believe  it  is  fixed. 

SATs  do  not  address  epistemological  beliefs.  As  with  reading,  the  SAT  may  augment  your 
knowledge  in  a  broad  way,  improving  the  content  of  your  mind,  and  facilitating  subsequent 
learning.  We  will  return  to  the  topic  of  epistemological  beliefs  later  in  this  FAC  section. 

8.3.2.  Types  of  Thinking  in  the  Cognitive  Domain 

As  mentioned  earlier,  the  cognitive  domain  is  where  meaning  is  assigned  to  information, 
problems  are  solved,  or  decisions  are  made.  Again,  while  little  agreement  exists  on  how  this 
thinking  is  accomplished,  some  agreement  has  been  reached  that  three  types  of  thinking  result  in 
“quality”  thinking  and  problem  solving:  critical  thinking,  reflective  thinking,  and  higher  order 
thinking.  These  three  types  of  thinking  occur  in  all  of  the  cognitive  domain  functions  (i.e., 
reading,  learning,  knowledge,  comprehension,  application,  analysis,  synthesis,  and  evaluation). 
They  are  of  interest  and  importance  to  our  consideration  of  the  cognition  of  analysis  and  SATs. 
After  we  review  these  three  types  of  thinking,  we’ll  summarize  the  contribution  of  SATs  to 
critical,  reflective,  and  higher  order  thinking. 

Critical  Thinking:  The  intelligence  literature  is  replete  with  views  that  using  critical  thinking 
will  resolve  most  if  not  all  the  issues  associated  with  the  quality  of  analysis.  Several  intelligence 
organizations  have  specific  training  programs  directed  at  developing  critical  thinking  skills  in 
analysts. 

What  is  critical  thinking?  The  definitions  are  numerous  and  not  always  consistent  with  one 
another,  but  here  is  the  one  developed  by  a  panel  of  experts  in  1990. 

We  understand  critical  thinking  to  be  purposeful,  self-regulatory  judgment  which  results 
in  interpretation,  analysis,  evaluation,  and  inference,  as  well  as  explanation  of  the 


mSchommer,  M.  "An  Emerging  Conceptualization  of  Epistemological  Beliefs  and  their  Role  in  Learning,"  in 
Beliefs  about  Texts  and  Instruction  with  Text,  eds.  R.  Garner  &  P.A.  Alexander,  L.  Erlbaum,  Hillsdale,  NJ,  1994,  p. 
28. 

112Schommer  p.  29. 

113Schommerpp.  34-35. 
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evidential,  conceptual,  methodological,  criteriological,  or  contextual  considerations  upon 
which  that  judgment  is  based.  CT  is  essential  as  a  tool  of  inquiry.114  [emphasis  added] 

These  experts  also  said: 

The  ideal  critical  thinker  is  habitually  inquisitive,  well-informed,  trustful  of  reason, 
open-minded,  flexible,  fair-minded  in  evaluation,  honest  in  facing  personal  biases, 
prudent  in  making  judgments,  willing  to  reconsider,  clear  about  issues,  orderly  in 
complex  matters,  diligent  in  seeking  relevant  information,  reasonable  in  the  selection  of 
criteria,  focused  in  inquiry,  and  persistent  in  seeking  results  which  are  as  precise  as  the 
subject  and  the  circumstances  of  inquiry  permit.1 15  [emphasis  added] 

The  experts  then  developed  a  listing  of  thinking  dispositions,  shown  in  Supplement  1  to  this 
section,  and  cognitive  skills,  shown  in  Supplement  2  to  this  section,  that  are  associated  with 
critical  thinking.  You  may  recall  that  we  discussed  thinking  dispositions  in  “Analysts  as 
Individuals,  Thinking  about  Thinking.”  Supplement  1  complements  that  information.  The 
listings  in  the  supplements  are  largely  descriptive,  focusing  on  what  to  do,  not  how  to  do  it. 
However,  we  recommend  thoughtful  consideration  of  your  own  behavior  with  regards  to  the 
cognitive  skills  and  thinking  dispositions  we  describe  in  the  supplements.  Doing  so  provides  you 
with  insight  into  your  thinking;  this  is  useful  preparation  for  the  next  section  of  the  FAC, 
“Assignment  of  Meaning.” 

Reflective  Thinking:  While  the  terms  critical  thinking  and  reflective  thinking  are  often  treated 
as  synonyms,  the  proponents  of  reflective  thinking  attribute  major  differences  between  the  two. 
Their  view  is  that  critical  thinking  is  more  focused  on  the  solution  of  well-structured  problems, 
while  reflective  thinking  is  better  suited  for  solving  ill-structured  problems.  This  distinction  is 
pertinent  the  topic  of  intelligence  analysis. 


114Facione,  P.A.  Critical  Thinking:  A  Statement  of  Expert  Consensus  for  Purposes  of  Educational  Assessment  and 
Instruction.  Research  Findings  and  Recommendations.  American  Philosophical  Association,  Newark,  DE,  1990,  p. 
3. 

115  Facione  p.  3. 
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A  NOTE  ABOUT  PROBLEM  SOLVING 

In  the  FAC  section  “Developing  Intelligence  Issue  Statements”  we  implied  that  addressing  an 
intelligence  issue  was  problem  solving,  but  we  didn’t  make  it  explicit.  This  language  was 
deliberate.  We  wanted  to  avoid  an  early  mindset  that  intelligence  issues  =  problems  since 
some  think  problems  usually  have  “right”  answers.  We  didn’t  want  to  imply  that  a  “right” 
answer  exists  for  every  intelligence  issue.  Finding  a  single  right  answer  is  a  very  simplified 
view  of  problems  and  their  solutions,  which  could  influence  how  you  understand  and  address 
the  intelligence  issue.  In  reality,  problems  have  a  great  diversity  of  character;  for  many,  no 
“right”  answer  exists. 

Using  this  more  complex  view  of  problems,  understanding  the  problem  is  what  we  discussed 
about  understanding  the  intelligence  issue.  In  general,  problems  are  categorized  as  either 
well-structured  or  unstructured.  Well-structured  problems  can  be  described  with  a  high  degree 
of  completeness,  can  be  solved  with  a  high  degree  of  certainty,  and  general  agreement  exists 
on  a  correct  solution.  In  contrast,  unstructured  problems  “cannot  be  described  with  a  high 
degree  of  completeness  or  solved  with  a  high  degree  of  certainty”  and  “it  is  sometimes 
difficult  to  determine  when  a  solution  has  been  reached.” 

Intelligence  issues  can  be  categorized  as  either  well-structured  or  unstructured  problems.  For 
example,  determining  the  order  of  battle  of  a  foreign  country  is  a  structured  problem,  but 
assessing  the  actions  of  a  foreign  government  in  2020  is  an  unstructured  problem.  So  analysis 
perfonned  on  an  intelligence  issue  is  indeed  a  form  of  problem  solving.  Relating  intelligence 
analysis  to  this  more  complex  view  of  problem  solving  is  important  since  a  substantial 
amount  of  research  has  been  perfonned  on  the  factors  that  affect  performance  on  problem 
solving.  Consequently,  the  results  of  such  research  can  provide  information  on  the  selection 
of  methods  and  techniques  that  will  aid  analysis. 

As  mentioned  in  earlier  sections,  the  FAC  is  focused  on  understanding  both  capabilities  and 
intentions  as  shaped  by  human  decisions  and  actions.  This  type  of  analysis  is  usually 
associated  with  unstructured  problems/intelligence  issues  so  the  research  results  on  solving 
such  problems  are  especially  pertinent  to  the  purpose  of  this  section. 

*  King,  P.M.  &  Kitchener,  K.S.  Developing  Reflective  Judgment:  Understanding  and  Promoting  Intellectual 
Growth  and  Critical  Thinking  in  Adolescents  and  Adults,  Jossey-Bass  Publishers,  San  Francisco,  CA,  1994,  pp. 
10-11. 


The  second  difference  between  critical  thinking  and  reflective  thinking  is  the  role  of  what  some 
refer  to  as  epistemological  assumptions.116  On  closer  examination,  epistemological  assumptions 
are  the  same  as  what  others  call  epistemological  beliefs.  As  with  learning,  one’s  ability  for 


U6King,  P.M.  &  Kitchener,  K.S.  Developing  Reflective  Judgment:  Understanding  and  Promoting  Intellectual 
Growth  and  Critical  Thinking  in  Adolescents  and  Adults,  Jossey-Bass  Publishers,  San  Francisco,  CA,  1994,  p.  8. 
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reflective  judgments  relates  to  one’s  state  of  epistemological  beliefs.  Naive  beliefs  result  in  poor 
judgments  and  sophisticated  beliefs  produce  more  nuanced  judgments.  Other  research  supports 
the  concept  that  the  ability  to  solve  unstructured  or  ill-structured  problems  depends  on 
epistemological  beliefs.117  118 

Higher  Order  Thinking:  As  with  critical  thinking,  many  definitions  are  used  for  higher  order 
thinking.  We  found  the  following  the  most  infonnative. 

Higher-order  thinking  is  the  mental  engagement  with  ideas,  objects,  and  situations  in  an 
analogical,  elaborative,  inductive,  deductive,  and  otherwise  transformational  manner  that 
is  indicative  of  an  orientation  toward  knowing  as  a  complex,  effortful,  generative, 
evidence-seeking,  and  reflective  enterprise.1 19  [emphasis  added] 

Basically,  higher  order  thinking  occurs  in  all  of  the  functions  we  discussed  within  the  cognitive 
domain.  Many  of  the  cognitive  skills  associated  with  higher  order  thinking  are  the  same  as  those 
for  critical  thinking.  However,  personal  attributes  are  given  more  attention  in  the  discussion  of 
higher  order  thinking. 

Precisely  because  higher-order  thinking,  as  we  and  others  have  defined  it,  requires 
effortful  and  unconventional  processing,  it  will  not  occur  unless  individuals  intend  to 
engage  mentally  at  the  requisite  level.  Thus,  any  conception  of  higher-order  thinking 
must  accept  the  necessary  willingness  of  the  individual  to  think  deeply,  analytically,  and 
in  an  otherwise  non-routine  manner.  [emphasis  added] 

Again,  we  see  the  importance  being  motivated  and  willing  to  engage  with  ideas,  as  we  did  with 
reading  and  learning. 

In  summary,  the  following  are  important  for  all  three  types  of  thinking  that  affect  the  quality  of 
thinking  and  problem  solving:  the  state  of  your  epistemological  beliefs;  having  knowledge  and 
goals;  and  being  well-informed,  purposeful,  orderly,  engaged,  and  persistent.  These  are  also 
important  for  reading  and  learning.  Our  review  of  the  SATs  indicates  that  they  can  help  you  be 
more  orderly  in  your  review  of  the  information,  but  SATs  do  not  address  the  state  of  your 
epistemological  beliefs.  In  addition,  much  as  for  reading  and  learning,  the  other  benefits  are 
indirect;  that  is,  the  SATs  may  generally  improve  your  knowledge  which  should  improve  all  of 
your  analytic  endeavors. 

8.3.3.  Cognitive  Factors  that  Affect  Thinking  Quality 

Our  starting  premise  was  that  an  understanding  of  the  cognition  associated  with  intelligence 
analysis  is  essential  for  you  to  consider  and  select  appropriate  SATs  to  use  as  you  assign 
meaning  to  information.  While  there  are  many  unknowns  about  how  people  think  and  various 
terms  are  used,  the  preceding  discussion  indicates  the  following  factors  are  important. 


ll7Harteis,  C.,  Gmber,  H.  &  Lehner,  F.  "Epistemological  Beliefs  and  their  Impact  on  Work,  Subjectivity,  and 
Learning,"  in  Work,  Subjectivity  and  Learning:  Understanding  Learning  through  Working  Life,  eds.  S.  Billett,  T.J. 
Fenwick  &  M.  Somerville,  Springer,  Dordrecht,  The  Netherlands,  2006,  p.  126. 

U8Schommer,  M.  "Synthesizing  Epistemological  Belief  Research:  Tentative  Understandings  and  Provocative 
Confusions,"  Educational  Psychology’  Review,  Vol.  6,  No.  4,  1994,  p.  305. 

"Alexander,  P.A.,  Dinsmore,  D.L.,  Fox,  E.,  Grossnickle,  E.M.,  Loughlin,  S.M.,  Maggioni,  L.,  Parkinson,  M.M.  & 
Winters,  FT.  Higher-Order  Thinking  and  Knowledge:  Domain-General  and  Domain-Specific  Trends  and  Future 
Directions.  Manuscript,  p.  11. 

"Alexander  p.  22. 
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•  Knowledge 

•  Thinking  dispositions 

•  Cognitive  skills 

•  Motivation  and  engagement 

•  Epistemological  beliefs 

These  factors  correlate  very  well  with  the  propensities  of  the  reflective  and  algorithmic  minds 
responsible  for  Type  2  processing  (deep  thinking)  that  we  discussed  in  “Analysts  as  Individuals” 
and  “Thinking  about  Thinking.” 

As  we  discussed,  the  propensities  of  the  reflective  mind  relate  to  thinking  dispositions  and  the 
inputs  to  the  mindware.  These  mindware  inputs  include: 

•  Knowledge 

•  Beliefs 

•  Goals 

The  algorithmic  mind  includes  the  strategies  for  processing  knowledge,  referred  to  as  procedural 
knowledge. 

To  summarize,  both  the  list  of  factors  that  impact  the  quality  of  thinking  and  the  inputs  to  the 
reflective  mind  have  some  common  themes.  Both  lists  contain  knowledge.  Strategies  for 
processing  knowledge  relate  closely  to  cognitive  skills  and  thinking  dispositions.  Goals  relate  to 
one’s  motivation  and  engagement. 

While  the  factor,  beliefs,  is  not  included  on  our  initial  list  of  cognitive  factors  affecting  thinking, 
for  thousands  of  years  the  philosophers  have  been  arguing  about  the  commonality  and  difference 
between  knowledge  and  beliefs,  resulting  for  some  in  interchangeable  use  of  the  two  terms. 
Based  on  these  discussions  about  knowledge  and  beliefs,  it  is  appropriate  to  add  beliefs  to  our 
list  of  factors  that  have  a  major  impact  on  the  quality  of  thinking.  Distilling  the  two  lists  to  a  set 
of  primary  cognitive  factors  that  impact  thinking  quality,  we  have: 

Cognitive  Factors  that  Impact  Thinking  Quality 

•  Knowledge 

•  Beliefs 

•  Goals 

•  Thinking  dispositions 

•  Cognitive  skills 

•  Epistemological  beliefs 

As  you  consider  which  SATs  to  use,  you  should  evaluate  the  SATs  against  these  factors  to 
determine  which  ones  aid  may  analytic  cognition  and  how  they  improve  it. 

In  addition,  we  mentioned  the  importance  of  the  problem  solving  literature  in  your  consideration 
of  SATs.  This  literature  emphasizes  the  nature  of  the  problem  (i.e.,  well-structured  or  ill- 
structured).  Many  SATs  attempt  to  add  structure  to  problems.  If  problems  are  relatively  well- 
structured,  this  may  be  useful.  However,  if  they  are  very  ill-structured,  adding  a  structure  that 
does  not  reflect  the  essence  of  the  problem  may  be  counterproductive.  The  value  of  using  the 
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SAT  may  then  be  related  to  how  useful  it  is  to  treat  your  issue  in  a  more  structured  way.  The 
literature  also  emphasizes  the  importance  of  understanding  the  problem  to  be  solved,  which  is 
the  goal  of  thinking. 

8.4  Categorization  of  SATs  Based  on  the  Cognition  of  Analysis 

As  mentioned  at  the  beginning  of  this  section,  a  great  deal  of  emphasis  has  been  placed  on  the 

use  of  SATs.  Johnston  states  that  “Although  well  over  160  analytic  methods  are  available  to 

121 

intelligence  analyst,  few  methods  specific  to  the  domain  of  intelligence  analysis  exist.” 
However,  he  did  not  specifically  identify  these  160  plus  techniques  nor  has  the  intelligence 
literature.  Both  CIA  and  DIA  have  training  courses  on  SATs. 

Many  categorizations  of  SATs  have  been  done.  For  example,  Richards  Heuer  describes  eight 
categories:  Idea  Generation,  Scenarios  and  Indicators,  Hypothesis  Generation  and  Testing, 
Assessment  of  Cause  and  Effect,  Challenge  Analysis,  Conflict  Management,  and  Decision 
Support.  Heuer’s  recent  book  describes  50  different  techniques.  “  Based  on  our  summarization 
of  the  cognitive  factors  that  impact  thinking,  we  have  grouped  a  sampling  of  SATs  from  the 
intelligence  literature  (i.e.,  DIA,  CIA  courses,  and  some  others)  into  four  general  categories  in 
Supplement  3  to  this  section  to  aid  you  in  considering  the  use  of  SATs.  Of  course,  there  is  not  a 
direct  one-for-one  mapping  of  the  cognitive  factors  and  the  SATs.  In  addition,  many  SATs 
address  multiple  aspects  of  thinking  and  some  are  designed  for  individual  use,  some  for  group 
use. 


•  Pattern  Recognition:  Arraying  the  information  so  as  to  enhance  the  recognition  of 
patterns  or  relationships 

•  Changing  or  Validating  the  Mindware:  Challenging  the  analyst  to  rethink  the  basis 
upon  which  he  or  she  has  made  judgments 

•  Diverse  Mindware:  Adding  a  diversity  of  knowledge  and  beliefs  to  the  analysis 
process 

•  Predictive:  Predicting  future  events 

8.5  Assigning  Meaning:  Individually  and  In  the  Aggregate 

A  key  provision  of  the  FAC  is  assigning  meaning  to  all  relevant  information,  both  individually 
and  in  the  aggregate.  Consequently,  the  usefulness  of  the  SATs  for  assigning  meaning  to 
individual  pieces  of  information  as  well  as  to  the  aggregation  of  information  becomes  a  key 
consideration  in  your  consideration  of  the  SATs. 

Assigning  meaning  to  individual  pieces  of  infonnation  relates  to  what  Kent  says  should  occur 
after  all  the  relevant  information  has  been  acquired:  critical  evaluation  of  the  data  thus  assembled 
(step  4).  In  discussing  Step  4,  Kent  emphasizes  the  need  to  evaluate  the  individual  information 


121Johnston,  R.  Analytic  Culture  in  the  US  Intelligence  Community:  An  Ethnographic  Study,  Center  for  the  Study  of 
Intelligence,  Washington,  D.  C.,  2005,  p.  72. 

122Heuer,  R.J.  &  Pherson,  R.H.  Structured  Analytic  Techniques  for  Intelligence  Analysis,  CQ  Press  College, 
Washington,  D.C.,  2010. 


137 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 
88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


before  it  becomes  part  of  the  aggregation  of  infonnation  from  which  a  hypothesis  will  eventually 

123 

emerge. 

With  a  few  exceptions,  in  the  intelligence  literature  most  SATs  are  discussed  in  terms  of 
evaluating  the  aggregation  of  information,  not  the  individual  pieces  of  information.  During  your 
evaluation  of  the  individual  pieces  of  infonnation,  you  will  engage  in  all  of  the  cognitive 
functions  (i.e.,  reading,  learning,  knowledge,  comprehension,  application,  analysis,  synthesis, 
and  evaluation).  Because  most  SATs  are  not  discussed  for  use  on  the  individual  pieces  of 
information,  a  major  portion  of  your  cognition  during  analysis  is  not  addressed.  Conceptually 
though,  some  of  the  SATs  could  be  used  in  your  evaluation  of  the  individual  pieces  of 
information  in  relationship  to  one  another.  Those  SATs  that  aid  you  with  pattern  recognition 
(e.g.,  developing  timelines)  are  good  examples. 

The  other  three  categories  of  SATs  (Changing  or  Validating  the  Mindware,  Diverse  Mindware, 
and  Predictive)  are  mostly  applicable  to  the  next  step  outlined  by  Kent:  study  of  the  evaluated 
data  with  the  intent  of  finding  some  inherent  meaning  (Step  5).  Step  5  refers  to  your  assignment 
of  meaning  to  the  aggregation  of  the  information.  You  will  also  engage  in  all  of  the  cognitive 
functions  as  you  assign  of  meaning  to  the  aggregation  of  information. 

You  should  also  consider  that  some  SATs  do  not  focus  on  the  individual  analyst  as  he  or  she 
reviews  the  set  of  relevant  information  (both  individual  pieces  and  the  aggregation).  Instead, 
these  SATs  more  generally  address  the  general  state  of  mind  of  the  analysts  as  they  work  in 
groups  together  (e.g.,  those  in  the  Diverse  Mindware  category). 

8.6  Summary 

This  section  has  provided  you  with  additional  information  about  thinking  to  assist  you  as  you 
prepare  to  assign  meaning  to  the  information  associated  with  your  intelligence  issue.  Research 
indicates  that  the  cognitive  factors  identified  can  impact  the  quality  of  your  thinking. 
Consequently,  as  you  do  your  analysis  it  is  important  for  you  to  deliberately  consider  the  state  of 
your  knowledge,  your  beliefs  and  goals,  the  thinking  dispositions  you  use  to  approach  your 
analysis,  and  your  overall  beliefs  about  the  nature  of  knowledge  (epistemological  beliefs).  As 
we  introduced  in  the  “Analysts  as  Individuals”  section,  deliberate  consideration  and  reflection 
about  your  thinking  throughout  your  analysis  are  essential  to  improve  your  judgments  about  the 
meaning  of  information. 

We  have  also  discussed  the  increasing  attention  in  the  intelligence  literature  on  the  role  of  SATs. 
Proponents  think  that  they  will  improve  the  quality  of  analysis,  owever,  little  has  been  done  to 
systematically  assess  the  potential  benefits  of  these  SATs.  hether  SATs  will  benefit  you  in  your 
particular  situation  is  something  that  you  should  consider  on  a  case-by-case  basis.  This  section 
has  provided  you  with  background  information  about  analysis  and  cognition  intended  to  aid  you 
as  you  consider  the  use  of  SATs  or  select  SATs  to  use  on  a  particular  intelligence  issue.  Key 
factors  you  should  consider  are:  the  nature  of  the  intelligence  issue,  how  the  SAT  aids  the 
cognition  of  analysis,  if  you  are  working  individually  or  in  a  group  (or  both),  and  whether  you 
want  to  use  the  SAT  to  assign  meaning  to  individual  pieces  of  infonnation  or  the  aggregation  of 
information. 


123Kent,  S.  Strategic  Intelligence  for  American  World  Policy,  Princeton  University  Press,  Princeton,  NJ,  1949,  p. 
169. 
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The  preceding  review  of  the  cognition  of  analysis  indicates  that  while  some  SATs  may  improve 
the  ability  to  structure  or  deal  with  the  information  in  a  more  orderly  way,  generally  they  provide 
only  indirect  benefits  to  aid  your  overall  development  of  knowledge.  Since  not  many  SATs 
address  assigning  meaning  to  the  individual  pieces  of  information,  the  next  FAC  section 
Assignment  of  Meaning  will  discuss  this  subject,  nee  the  individual  pieces  of  relevant 
information  have  been  evaluated,  the  use  of  additional  analysis  techniques  and  methods  for  the 
aggregated  set  of  evaluated  relevant  information  will  be  described  in  more  detail  in  subsequent 
sections  of  the  FAC. 
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8.7  Supplement  1:  Thinking  Dispositions124 

APPROACHES  TO  LIFE  AND  LIVING  IN  GENERAL: 

•  inquisitiveness  with  regard  to  a  wide  range  of  issues, 

•  concern  to  become  and  remain  well-informed, 

•  alertness  to  opportunities  to  use  critical  thinking, 

•  trust  in  the  processes  of  reasoned  inquiry, 

•  self-confidence  in  one’s  own  abilities  to  reason, 

•  open-mindedness  regarding  divergent  world  views, 

•  flexibility  in  considering  alternatives  and  opinions 

•  understanding  of  the  opinions  of  other  people, 

•  fair-mindedness  in  appraising  reasoning, 

•  honesty  in  facing  one’s  own  biases,  prejudices,  stereotypes,  or  egocentric 
tendencies, 

•  prudence  in  suspending,  making  or  altering  judgments, 

•  willingness  to  reconsider  and  revise  views  where  honest  reflection  suggests  that 
change  is  warranted. 

APPROACHES  TO  SPECIFIC  ISSUES,  QUESTIONS,  OR  PROBLEMS: 

•  clarity  in  stating  the  question  or  concern, 

•  orderliness  in  working  with  complexity, 

•  diligence  in  seeking  relevant  information, 

•  reasonableness  in  selecting  and  applying  criteria, 

•  care  in  focusing  attention  on  the  concern  at  hand, 

•  persistence  though  difficulties  are  encountered, 

•  precision  to  the  degree  permitted  by  the  subject  and  the  circumstances. 


124 

Extracted  from  Facione,  P.A.  Critical  Thinking:  A  Statement  of  Expert  Consensus  for  Purposes  of  Educational 
Assessment  and  Instruction.  Research  Findings  and  Recommendations.  American  Philosophical  Association, 
Newark,  DE,  1990,  p.  25. 
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8.8  Supplement  2:  Cognitive  Skills125 
8.8.1.  Interpretation 

To  comprehend  and  express  the  meaning  or  significance  of  a  wide  variety  of  experiences, 
situations,  data,  events,  judgments,  conventions,  beliefs,  rules,  procedures  or  criteria. 

Categorization: 

•  to  apprehend  or  appropriately  formulate  categories,  distinctions,  or  frameworks  for 
understanding,  describing  or  characterizing  infonnation. 

•  to  describe  experiences,  situations,  beliefs,  events,  etc.  so  that  they  take  on 
comprehensible  meanings  in  terms  of  appropriate  categorizations,  distinctions,  or 
frameworks. 

For  example:  to  recognize  a  problem  and  define  its  character  without  prejudice  to  inquiry;  to 
determine  a  useful  way  of  sorting  and  sub-classifying  information;  to  make  an  understandable 
report  of  what  one  experienced  in  a  given  situation;  to  classify  data,  findings  or  opinions  using  a 
given  classification  schema. 

Decoding  Significance: 

•  to  detect,  attend  to,  and  describe  the  informational  content,  affective  purport, 
directive  functions,  intentions,  motives,  purposes,  social  significance,  values,  views, 
rules,  procedures,  criteria,  or  inferential  relationships  expressed  in  convention-based 
communication  systems,  such  as  in  language,  social  behaviors,  drawings,  numbers, 
graphs,  tables,  charts,  signs  and  symbols. 

For  example:  to  detect  and  describe  a  person's  purposes  in  asking 

a  given  question;  to  appreciate  the  significance  of  a  particular  facial  expression  or  gesture  used  in 
a  given  social  situation;  to  discern  the  use  of  irony  or  rhetorical  questions  in  debate;  to  interpret 
the  data  displayed  or  presented  using  a  particular  form  of  instrumentation. 

Clarifying  Meaning : 

•  to  paraphrase  or  make  explicit,  through  stipulation,  description,  analogy  or 
figurative  expression,  the  contextual,  conventional  or  intended  meanings  of  words, 
ideas,  concepts,  statements,  behaviors,  drawings,  numbers,  signs,  charts,  graphs, 
symbols,  rules,  events  or  ceremonies. 

•  to  use  stipulation,  description,  analogy  or  figurative  expression  to  remove 
confusing,  unintended  vagueness  or  ambiguity,  or  to  design  a  reasonable  procedure 
for  so  doing. 

For  example:  to  restate  what  a  person  said  using  different  words  or  expressions  while  preserving 
that  person's  intended  meanings;  to  find  an  example  which  helps  explain  something  to  someone; 
to  develop  a  distinction  which  makes  clear  a  conceptual  difference  or  removes  a  troublesome 
ambiguity. 


125  Extracted  from  Facione  pp.  13-19. 
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8.8.2.  Analysis 

To  identify  the  intended  and  actual  inferential  relationships  among  statements,  questions, 
concepts,  descriptions  or  other  forms  of  representation  intended  to  express  beliefs,  judgments, 
experiences,  reasons,  information,  or  opinions. 

Examining  Ideas: 

•  to  detennine  the  role  various  expressions  play  or  are  intended  to  play  in  the  context 
of  argument,  reasoning  or  persuasion. 

•  to  define  terms. 

•  to  compare  or  contrast  ideas,  concepts,  or  statements. 

•  to  identify  issues  or  problems  and  determine  their  component  parts,  and  also  to 
identify  the  conceptual  relationships  of  those  parts  to  each  other  and  to  the  whole. 

For  example:  to  identify  a  phrase  intended  to  trigger  a  sympathetic  emotional  response  which 
might  induce  an  audience  to  agree  with  an  opinion;  to  examine  closely  related  proposals 
regarding  a  given  problem  and  to  detennine  their  points  of  similarity  and  divergence;  given  a 
complicated  assignment,  to  determine  how  it  might  be  broken  up  into  smaller,  more  manageable 
tasks;  to  define  an  abstract  concept. 

Detecting  Arguments: 

•  given  a  set  of  statements,  descriptions,  questions  or  graphic  representations,  to 
determine  whether  or  not  the  set  expresses,  or  is  intended  to  express,  a  reason  or 
reasons  in  support  of  or  contesting  some  claim,  opinion  or  point  of  view. 

For  example,  given  a  paragraph,  detennine  whether  a  standard  reading  of  that  paragraph  in  the 
context  of  how  and  where  it  is  published,  would  suggest  that  it  presents  a  claim  as  well  as  a 
reason  or  reasons  in  support  of  that  claim)  given  a  passage  from  a  newspaper  editorial,  determine 
if  the  author  of  that  passage  intended  it  as  an  expression  of  reasons  for  or  against  a  given  claim 
or  opinion;  given  a  commercial  announcement,  identify  any  claims  being  advanced,  along  with 
the  reasons  presented  in  their  support. 

Analyzing  Arguments: 

•  given  the  expression  of  a  reason  or  reasons  intended  to  support  or  contest  some 
claim,  opinion  or  point  of  view,  to  identify  and  differentiate:  (a)  the  intended  main 
conclusion,  (b)  the  premises  and  reasons  advanced  in  support  of  the  main 
conclusion,  (c)  further  premises  and  reasons  advanced  as  backup  or  support  for 
those  premises  and  reasons  intended  as  supporting  the  main  conclusion,  (d) 
additional  unexpressed  elements  of  that  reasoning,  such  as  intermediary 
conclusions,  unstated  assumptions  or  presuppositions,  (e)  the  overall  structure  of 
the  argument  or  intended  chain  of  reasoning,  and  (f)  any  items  contained  in  the 
body  of  expressions  being  examined  which  are  not  intended  to  be  taken  as  part  of 
the  reasoning  being  expressed  or  its  intended  background. 

For  example:  given  a  brief  argument,  paragraph-sized  argument,  or  a  position  paper  on  a 
controversial  social  issue,  to  identify  the  author's  chief  claim,  the  reasons  and  premises  the 
author  advances  on  behalf  of  that  claim,  the  background  information  used  to  support  those 
reasons  or  premises,  and  crucial  assumptions  implicit  in  the  author's  reasoning;  given  several 
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reasons  or  chains  of  reasons  in  support  of  a  particular  claim,  to  develop  a  graphic  representation 
which  usefully  characterizes  the  inferential  flow  of  that  reasoning. 

8.8.3.  Evaluation 

To  assess  the  credibility  of  statements  or  other  representations  which  are  accounts  or  descriptions 
of  a  person's  perception,  experience,  situation,  judgment,  belief,  or  opinion;  and  to  assess  the 
logical  strength  of  the  actual  or  intend  inferential  relationships  among  statements,  descriptions, 
questions  or  other  forms  of  representation. 

Assessing  Claims: 

•  to  recognize  the  factors  relevant  to  assessing  the  degree  of  credibility  to  ascribe  to  a 
source  of  information  or  opinion. 

•  to  assess  the  contextual  relevance  of  questions,  information,  principles,  rules  or 
procedural  directions. 

•  to  assess  the  acceptability,  the  level  of  confidence  to  place  in  the  probability  or  truth 
of  any  given  representation  of  an  experience,  situation,  judgment,  belief  or  opinion. 

For  example:  to  recognize  the  factors  which  make  a  person  a  credible  witness  regarding  a  given 
event  or  credible  authority  on  a  given  topic;  to  determine  if  a  given  principle  of  conduct  is 
applicable  to  deciding  what  to  do  in  a  given  situation;  to  detennine  if  a  given  claim  is  likely  to  be 
true  or  false  based  on  what  one  knows  or  can  reasonably  find  out. 

Assessing  Arguments: 

•  to  judge  whether  the  assumed  acceptability  of  the  premises  of  a  given  argument 
justify  one's  accepting  as  true  (deductively  certain),  or  very  probably  true 
(inductively  justified),  the  expressed  conclusion  of  that  argument. 

•  to  anticipate  or  to  raise  questions  or  objections,  and  to  assess  whether  these  point  to 
significant  weakness  in  the  argument  being  evaluated. 

•  to  detennine  whether  an  argument  relies  on  false  or  doubtful  assumptions  or 
presuppositions  and  then  to  detennine  how  crucially  these  affect  its  strength. 

•  to  judge  between  reasonable  and  fallacious  inferences; 

•  to  judge  the  probative  strength  of  an  argument's  premises  and  assumptions  with  a 
view  toward  detennining  the  acceptability  of  the  argument. 

•  to  detennine  and  Judge  the  probative  strength  of  an  argument's  intended  or 
unintended  consequences  with  a  view  toward  judging  the  acceptability  of  the 
argument; 

•  to  determine  the  extent  to  which  possible  additional  information  might  strengthen  or 
weaken  an  argument. 

For  example:  given  an  argument  to  judge  if  its  conclusions  follow  either  with  certainty  or  with  a 
high  level  of  confidence  from  it  premises,  to  check  for  identifiable  fonnal  and  informal  fallacies; 
given  an  objection  to  an  argument  to  evaluate  the  logical  force  of  the  objection;  to  evaluate  the 
quality  and  applicability  of  analogical  arguments;  to  judge  the  logical  strength  of  arguments 
based  on  hypothetical  situations  of  causal  reasoning;  to  judge  if  a  given  argument  is  relevant  of 
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applicable  or  has  implications  for  the  situation  at  hand;  to  detennine  how  possible  new  data 
might  lead  logically  to  the  further  confirmation  of  disconfirmation  of  the  given  opinion. 

8.8.4.  Inference 

To  identify  and  secure  elements  needed  to  draw  reasonable  conclusions;  to  form  conjectures  and 
hypotheses;  to  consider  relevant  infonnation  and  to  educe  the  consequences  flowing  from  data, 
statements,  principles,  evidence,  judgments,  beliefs,  opinions,  concepts,  descriptions,  questions, 
or  other  fonns  of  representation. 

Querying  Evidence: 

•  in  particular,  to  recognize  premises  which  require  support  and  to  formulate  a 
strategy  for  seeking  and  gathering  Information  which  might  supply  that  support. 

•  in  general,  to  judge  that  information  relevant  to  deciding  the  acceptability, 
plausibility  or  relative  merits  of  a  given  alternative,  question,  issue,  theory, 
hypothesis,  or  statement  is  required,  and  to  determine  plausible  investigatory 
strategies  for  acquiring  that  infonnation. 

For  example:  when  attempting  to  develop  a  persuasive  argument  in  support  of  one’s  opinion,  to 
judge  what  background  information  it  would  be  useful  to  have  and  to  develop  a  plan  which  will 
yield  clear  answer  as  to  whether  or  not  such  information  is  available;  after  judging  that  certain 
missing  information  would  be  gennane  in  detennining  if  a  given  opinion  is  more  or  less 
reasonable  than  a  competing  opinion,  to  plan  a  search  which  will  reveal  if  that  information  is 
available. 

Conjecturing  Alternatives : 

•  to  formulate  multiple  alternatives  for  resolving  a  problem,  to  postulate  a  series  of 
suppositions  regarding  a  question,  to  project  alternative  hypotheses  regarding  an 
event,  to  develop  a  variety  of  different  plans  to  achieve  some  goal. 

•  to  draw  out  presuppositions  and  project  the  range  of  possible  consequences  of 
decisions,  positions,  policies,  theories,  or  beliefs. 

For  example:  given  a  problem  with  technical,  ethnical  or  budgetary  ramifications,  to  develop  a 
set  of  options  for  addressing  and  resolving  that  problem;  given  a  set  of  priorities  with  which  one 
may  or  may  not  agree,  to  project  the  difficulties  and  the  benefits  which  are  likely  to  result  if 
these  priorities  are  adopted  in  decision  making. 

Drawing  Conclusions: 

•  to  apply  appropriate  modes  of  inference  in  determining  what  position,  opinion  or 
point  of  view  one  should  take  on  a  given  matter  or  issue. 

•  given  a  set  of  statements,  descriptions,  questions  or  other  forms  of  representation,  to 
educe,  with  the  proper  level  of  logical  strength,  their  inferential  relationships  and 
the  consequences  or  the  presuppositions  which  they  support,  warrant,  imply  or 
entail. 

•  to  employ  successfully  various  sub-species  of  reasoning,  as  for  example  to  reason 
analogically,  arithmetically,  dialectically,  scientifically,  etc. 
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•  to  determine  which  of  several  possible  conclusions  is  most  strongly  warranted  or 
supported  by  the  evidence  at  hand,  or  which  should  be  rejected  or  regarded  as  less 
plausible  by  the  information  given. 

For  example:  to  carry  out  experiments  and  to  apply  appropriate  statistical  inference  techniques  in 
order  to  confirm  or  disconfinn  an  empirical  hypothesis;  given  a  controversial  issue  to  examine 
informed  opinions,  consider  various  opposing  views  and  the  reasons  advanced  for  them,  gather 
relevant  infonnation,  and  formulate  one's  own  considered  opinion  regarding  that  issue;  to  deduce 
a  theorem  from  axioms  using  prescribed  rules  of  inference. 

8.8.5.  Explanation 

To  state  the  results  of  one's  reasoning;  to  justify  that  reasoning  in  terms  of  the  evidential, 
conceptual,  methodological,  criteriological  and  contextual  considerations  upon  which  one's 
results  were  based;  and  to  present  one's  reasoning  in  the  form  of  cogent  arguments. 

Stating  Results: 

•  to  produce  accurate  statements,  descriptions  or  representations  of  the  results  of 
one's  reasoning  activities  so  as  to  analyze,  evaluate,  infer  from,  or  monitor  those 
results. 

For  example:  to  state  one's  reasons  for  holding  a  given  view;  to  write  down  for  one's  own  future 
use  one's  current  thinking  about  an  important  or  complex  matter  to  state  one’s  research  findings; 
to  convey  one's  analysis  and  judgment  regarding  a  work  of  art;  to  state  one's  considered  opinion 
on  a  matter  of  practical  urgency. 

Justifying  Procedures: 

•  to  present  the  evidential,  conceptual,  methodological,  criteriological  and  contextual 
considerations  which  one  used  in  forming  one's  interpretations,  analyses,  evaluation 
or  inferences,  so  that  one  might  accurately  record,  evaluate,  describe  or  justify  those 
processes  to  one's  self  or  to  others,  or  so  as  to  remedy  perceived  deficiencies  in  the 
general  way  one  executes  those  processes. 

For  example:  to  keep  a  log  of  the  steps  followed  in  working  through  a  long  or  difficult  problem 
or  scientific  procedure;  to  explain  one's  choice  of  a  particular  statistical  test  for  purposes  of  data 
analysis;  to  state  the  standards  one  used  in  evaluating  a  piece  of  literature  to  explain  how  one 
understands  a  key  concept  when  conceptual  clarity  is  crucial  for  further  progress  on  a  given 
problem;  to  show  that  the  prerequisites  for  the  use  of  a  given  technical  methodology  have  been 
satisfied;  to  report  the  strategy  used  in  attempting  to  make  a  decision  in  a  reasonable  way;  to 
design  a  graphic  display  which  represents  the  quantitative  or  spatial  information  used  as 
evidence. 

Presenting  Arguments: 

•  to  give  reasons  for  accepting  some  claim. 

•  to  meet  objections  to  the  method,  conceptualizations,  evidence,  criteria  or 
contextual  appropriateness  of  inferential,  analytical  or  evaluative  judgments. 

For  example:  to  write  a  paper  in  which  one  argues  for  a  given  position  or  policy;  to  anticipate 
and  to  respond  to  reasonable  criticisms  one  might  expect  to  be  raised  against  one’s  political 
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views;  to  identify  and  express  evidence  and  counter-evidence  intended  as  a  dialectical 
contribution  to  one's  own  or  another  person's  thinking  on  a  matter  of  deep  personal  concern. 

8.8.6.  Self-Regulation 

Self-consciously  to  monitor  one's  cognitive  activities,  the  elements  used  in  those  activities,  and 
the  results  educed,  particularly  by  applying  skills  in  analysis  and  evaluation  to  one’s  own 
inferential  judgments  with  a  view  toward  questioning,  confirming,  validating,  or  correcting  either 
one's  reasoning  or  one's  results. 

Self-Examination: 

•  to  reflect  on  one's  own  reasoning  and  verify  both  the  results  produced  and  the 
correct  application  and  execution  of  the  cognitive  skills  involved. 

•  to  make  an  objective  and  thoughtful  meta-cognitive  self-assessment  of  one's 
opinions  and  reasons  for  holding  them. 

•  to  judge  the  extent  to  which  one's  thinking  is  influenced  by  deficiencies  in  one's 
knowledge,  or  by  stereotypes,  prejudices,  emotions  or  any  other  factors  which 
constrain  one’s  objectivity  or  rationality. 

•  to  reflect  on  one's  motivations,  values,  attitudes  and  interests  with  a  view  toward 
determining  that  one  has  endeavored  to  be  unbiased,  fair-minded,  thorough, 
objective,  respectful  of  the  truth,  reasonable,  and  rational  in  coming  to  one's 
analyses,  interpretations,  evaluations,  inferences,  or  expressions. 

For  example:  to  examine  one's  views  on  a  controversial  issue  with  sensitivity  to  the  possible 
influences  of  one's  personal  bias  or  self-interest;  to  review  one's  methodology  or  calculations 
with  a  view  to  detecting  mistaken  applications  or  inadvertent  errors;  to  reread  sources  to  assure 
that  one  has  not  overlooked  important  information;  to  identify  and  review  the  acceptability  of  the 
facts,  opinions  or  assumptions  one  relied  on  in  coining  to  a  given  point  of  view;  to  identify  and 
review  one's  reasons  and  reasoning  processes  in  coming  to  a  given  conclusion. 

Self-Correction: 

•  where  self-examination  reveals  errors  or  deficiencies,  to  design  reasonable 
procedures  to  remedy  or  correct,  if  possible,  those  mistakes  and  their  causes. 

For  example:  given  a  methodological  mistake  or  factual  deficiency  in  one's  work,  to  revise  that 
work  so  as  to  correct  the  problem  and  then  to  determine  if  the  revisions  warrant  changes  in  any 
position,  findings  or  opinions  based  thereon. 
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8.9  Supplement  3:  Sampling  of  SATs 


NOTE 

The  SAT  information  in  quotation  marks  has  been  extracted  from  the  sources  identified 
with  each  description.  The  recommendations  for  use  are  those  provided  in  the  original 
publications.  We  have  grouped  these  SATs  into  four  general  categories  as  described  in  the 
FAC  “Analysis  Techniques”  section. 


8.9.1.  Pattern  Recognition 

Pattern  recognition  is  the  arraying  of  information  so  as  to  enhance  the  recognition  of  patterns  or 
relationships. 

126 

Social  Network  Analysis 

Description:  “Social  network  analysis  is  an  analytical  technique  which  describes  and  maps 
relationships  between  individuals,  groups,  organizations,  or  resources. ..This  technique  attempts 
to  answer  the  question  "who  knows  whom?"  Social  network  analysts  uncover  emerging  and 
informal  communication  patterns  in  an  organization  by  mapping  these  relationships.  With  this 
insight,  analysts  can  then  attempt  to  predict  behavior  and  decision  making  within  a  social 
organization  and  evaluate  specific  courses  of  action  that  could  influence  the  members  of  a  social 
network  in  a  desirable  way.” 

Trend  Analysis127 

Description:  “Trend  analysis,  analysis  of  changes  over  time,  is  used  primarily  in  marketing, 
business  planning,  and  strategic  planning,  at  national  and  corporate  levels  around  the  world.  It 
includes  a  number  of  sub-methodologies:  historical  trend  analysis,  content  analysis,  cyclical 
pattern  analysis,  and  the  use  of  expert  opinions  called  Delphi  processes.  It  is  not  designed  to  be 
used  as  a  stand-alone  method,  but  it  can  be  useful  when  combined  with  other  approaches.” 

128 

Five  Forces  Analysis 

Description:  This  model  claims  “that  five  forces  influence  every  market  and  industry.”  The  five 
forces  are  “Threat  of  New  Entrants. .  .Power  of  Suppliers. .  .Power  of  Buyers. .  .Availability  of 
Substitutes... Jockeying  for  Position... While  the  Five  Forces  Analysis  originally  serves  as  a 
business  model,  its  unique  analysis  of  competition  has  a  number  of  applications.  The  political 
environment  surrounding  a  government  official  exemplifies  a  beneficial  non-industry  use.  By 
adjusting  the  ultimate  goal  of  the  target  from  maximizing  profit  to  suit  the  specific 
circumstances,  the  Five  Forces  method  yields  insightful  analysis  of  virtually  any  environment.” 


126  Weaver,  L.E.  “Social  Network  Analysis"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco 
&  D.E.  Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  103-116. 

127  Hill,  Z.  "Trend  Analysis"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  &  D.E.  Chido, 
Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  145-156. 

128Viens,  M.  "Five  Forces  Analysis"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  &  D.E. 
Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  131-144. 
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129 

Strengths,  Weaknesses,  Opportunities,  and  Threats  (SWOT)  Analysis 

Description:  “SWOT  is  a  useful  analytical  technique  for  conducting  an  environmental  scan  to 
determine  an  organization's  internal  and  external  situation.  SWOT  analysis  helps  strategists 
focus  on  the  key  issues  that  they  must  address  in  order  to  enhance  an  organization's  success.  It  is 
especially  useful  in  analyzing  an  entity's  current  situation  in  a  short  period  of  time.” 

Geographic  Analysis 130 

Description:  “Geographic  Analysis  is  used  to  display  data  visually  in  a  geographic  format. 
Geographic  Infonnation  Systems  (GIS)  are  used  to  convert  data  into  spatial  and  geographic  maps 
in  order  for  the  analyst  to  discern  patterns  and  correlations  of  such  data.” 

Timeline  Analysis131 

Description:  “Timeline  analysis  is  an  excellent  technique  for  revealing  patterns  and  relationships 
among  data... This  technique  supports  trend  analyses,  situational  assessments,  and  event 
predictions.” 

Cost  Benefit  Analysis  (CBA)132 

Description:  “CBA  is  a  framework  for  assessing  and  comparing  the  costs  and  benefits  of  an 
activity,  project,  or  policy  over  a  particular  period  of  time.”  The  technique  “is  frequently  used  in 
the  public  sector  to  analyze  policies  affecting  public  projects  in  the  areas  of  transportation, 
health,  criminal  justice,  defense,  education,  and  the  environment.” 

Sorting 133 

Description:  “A  basic  structuring  technique  for  grouping  infonnation  to  develop  insight  to 
facilitate  analysis.  This  technique  is  most  useful  for  reviewing  massive  data  stores  that  pertain  to 
an  intelligence  challenge.” 

Event  Trees134 

Description:  “Graphical  depiction  of  a  potential  temporal  sequence  of  events,  including  potential 
junctures  within  the  event  sequence 


129  Sharma,  A.  "SWOT  Analysis"  in  The  Analyst’s  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  &  D.E. 
Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  125-130. 

130  Fitzgerald,  F.  "Geographic  Analysis  and  Crime  Mapping"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J. 
Wheaton,  E.E.  Mosco  &  D.E.  Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp. 
88-94. 

131  Williams-Taliaferro,  E.L.  "Timeline  Analysis"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E. 
Mosco  &  D.E.  Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  63-70. 

132  Pate,  E.  "Cost-Benefit  Analysis"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  &  D.E. 
Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  21-28. 

133  Anonymous  A  Tradecraft  Primer:  Basic  Structured  Analytic  Techniques. 2nd  edn..  Defense  Intelligence  Agency 
(DIA),  Washington,  D.C.,  2009,  p.  33. 

134  Anonymous  A  Tradecraft  Primer:  Basic  Structured  Analytic  Techniques. 2nd  edn.,  Defense  Intelligence  Agency, 
Washington,  D.  C.,  2009,  p.  51. 
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8.9.2.  Changing  or  Validating  Mindware 

This  SAT  challenges  the  analyst  to  rethink  the  basis  upon  which  he  or  she  has  made  judgments. 
Alternative  Thinking  Techniques 135 

Description:  “Alternative  thinking  techniques. .  .are  ways  of  thinking  that  disrupt  the  typical 
linear,  logical  thinking  pattern,  and,  by  considering  the  problem  or  question  from  all  viewpoints 
and  developing  options  which  would  not  otherwise  be  considered,  arrives  at  an  answer  from  a 
different  angle...” 

Subjective  Probability136 

Description:  “A  quantitative  expression  of  an  analyst's  degree  of  belief  in  the  truth  of  a  statement 
relative  to  others  from  among  a  complete  set  of  alternative  possibilities...  Subjective  probabilities 
are  used  to  quantitatively  express  an  analyst's  overall  degree  of  belief  in  the  truth  of  a  statement 
or  hypothesis  where  the  total  belief  held  by  an  analyst  is  allocated  among  the  possibilities  (non¬ 
overlapping  hypotheses)  in  proportion  to  how  likely  each  answer  or  event  is  correct.” 

Quality  of  Information  Check137 

Description:  “Evaluates  completeness  and  soundness  of  available  information  sources... 
Weighing  the  validity  of  sources  is  a  key  feature  of  any  critical  thinking.  If  a  major  analytic 
assessment  is  planned,  analysts  should  individually  or  collectively  review  the  quality  of  their 
information  and  refresh  their  understanding  of  the  strengths  and  weaknesses  of  past  reporting  on 
which  an  analytic  line  rests.” 

High-Impact/Low  Probability  Analysis138 

Description:  “Highlights  a  seemingly  unlikely  event  that  would  have  major  policy  consequences 
if  it  happened. .  .High-Impact/Low-Probability  Analysis  is  a  contrarian  technique  that  sensitizes 
analysts  to  the  potential  impact  of  seemingly  low  probability  events  that  would  have  major 
repercussions  on  US  interests. .  .If  there  is  a  strongly  held  view  that  an  event  is  unlikely,  then 
postulating  precisely  the  opposite  should  not  be  difficult.” 

“What  If?”  Analysis139 

Description:  “Assumes  that  an  event  has  occurred  with  potential  (negative  or  positive)  impact 
and  explains  how  it  might  come  about... “What  If?”  analysis  is  another  contrarian  technique  for 
challenging  a  strong  mind-set  that  an  event  will  not  happen  or  that  a  confidently  made  forecast 
may  not  be  entirely  justified.” 


135  Wozny,  J.L.  "Alternative  Thinking  Techniques"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E. 
Mosco  &  D.E.  Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  29-34. 

136  Anonymous  A  Tradecraft  Primer:  Basic  Structured  Analytic  Techniques. 2nd  edn.,  Defense  Intelligence  Agency, 
Washington,  D.C.,  2009,  pp.  59-62. 

1,7AnonymousT  Tradecraft  Primer:  Structured  Analytic  Techniques  for  Improving  Intelligence  Analysis  2009.,  pp. 
10-11. 


Anonymous  pp.  22-23 
139  Anonymous  pp.  24-25 
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Outside-In  Thinking140 

Description:  “Used  to  identify  the  full  range  of  basic  forces,  factors,  and  trends  that  would 
indirectly  shape  an  issue.”  This  technique  can  be  .  .most  useful  at  the  conceptualization  of  an 
analytic  project,  when  the  goal  is  to  identify  all  the  critical,  external  factors  that  could  influence 
how  a  particular  situation  will  develop.” 

8.9.3.  Diversive  Mindware 

Diversive  mindware  adds  a  diversity  of  knowledge  and  beliefs  to  the  analysis  process. 

Event  Mapping 141 

Description  -“A  mind-mapping  diagram  representing  the  scenarios  in  hypotheses  linked  around  a 
central  word  or  short  phrase  representing  the  issue  or  problem  to  be  analyzed.” 

Weighted  Ranking 142 

Description  -  “A  technique  used  by  an  individual  or  group  to  gain  confidence  in  the  assessment 
of  available  alternatives  by  weighting  criteria  in  importance  from  the  decision  maker’s  point  of 
view.” 

Key  Assumptions  Check143 

Description  -  “List  and  review  the  key  working  assumptions  on  which  fundamental  judgments 
rest...  A  Key  Assumptions  Check  is  most  useful  at  the  beginning  of  an  analytic  project.  An 
individual  analyst  or  a  team  can  spend  an  hour  or  two  articulating  and  reviewing  the  key 
assumptions. .  .A  key  assumption  is  any  hypothesis  that  analysts  have  accepted  to  be  true  and 
which  forms  the  basis  of  the  assessment.” 

Indicators  or  Signposts  of  Change 144 

Description  -  “Periodically  review  a  list  of  observable  events  or  trends  to  track  events,  monitor 
targets,  spot  emerging  trends,  and  warn  of  unanticipated  change. .  .An  analyst  or  team  can  create 
an  indicators  or  signposts  list  of  observable  events  that  one  would  expect  to  see  if  a  postulated 
situation  is  developing.” 

Analysis  of  Competing  Hypotheses  (ACH)145 

Description  -  “Identification  of  alternative  explanations  (hypotheses)  and  evaluation  of  all 
evidence  that  will  disconfirm  rather  than  confirm  hypotheses...  While  a  single  analyst  can  use 
ACH,  it  is  most  effective  with  a  small  team  that  can  challenge  each  other’s  evaluation  of  the 
evidence... ACH  demands  that  analysts  explicitly  identify  all  the  reasonable  alternative 
hypotheses,  then  array  the  evidence  against  each  hypothesis — rather  than  evaluating  the 
plausibility  of  each  hypothesis  one  at  a  time.  ” 


140  Anonymous  p.  30 

141  Anonymous  p.  55. 

142  Anonymous  p.  63. 

143  Anonymous.  A  Tradecr aft  Primer:  Structured  Analytic  Techniques  for  Improving  Intelligence  Analysis  2009.  pp. 
7-9. 

Anonymous  p.  12. 

143  Anonymous  pp.  14-16. 
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Devil's  Advocacy146 : 

Description  -  “Challenging  a  single,  strongly  held  view  or  consensus  by  building  the  best 
possible  case  for  an  alternative  explanation. .  .Devil’s  Advocacy  is  most  effective  when  used  to 
challenge  an  analytic  consensus  or  a  key  assumption  regarding  a  critically  important  intelligence 
question. .  .designate  a  courageous  analyst  to  challenge  the  prevailing  wisdom  in  order  to  reaffirm 
the  group’s  confidence  in  those  judgments.” 

Team  A/Team  B147 

Description  -  “Use  of  separate  analytic  teams  that  contrast  two  (or  more)  strongly  held  views  or 
competing  hypotheses... A  Team  A/Team  B  approach  is  different  from  Devil’s  Advocacy,  where 
the  purpose  is  to  challenge  a  single  dominant  mind-set.” 

Brainstorming148 

Description  -  “An  unconstrained  group  process  designed  to  generate  new  ideas  and 
concepts. .  .Brainstorming  is  a  widely  used  technique  for  stimulating  new  thinking  and  it  can  be 
applied  to  virtually  all  of  the  other  structured  analysis  techniques  as  an  aid  to  thinking.” 

Red  Team  Analysis149: 

Description  -  “Models  the  behavior  of  an  individual  or  group  by  trying  to  replicate  how  an 
adversary  would  think  about  an  issue. .  .Red  Team  analysis  tries  to  consciously  place  analysts  in 
the  same  cultural,  organizational,  and  personal  setting  (“putting  them  in  their  shoes”)  in  which 
the  target  individual  or  group  operates.” 

Predictive  Techniques 

Predictive  techniques  are  used  to  foretell  future  events. 

Predictive  Markets150 

Description  -  “Predictive  markets  are  essentially  futures  markets... Predictive  markets  invent  an 
individual  security  and  then  create  a  market  for  this  security.  The  security  can  be  on  virtually  any 
subject,  with  the  key  being  the  type  of  subject  and  the  size  of  the  market.” 

Decision  Trees151 

Description  -  “Decision  trees  are  excellent  tools  for  classification  and  prediction  and  allow  for  a 
more  thorough  analysis  of  a  decision  making  dilemma.  Through  a  series  of  nodes  and  branches, 
decision  trees  provide  a  highly  effective  structure  that  illustrates  the  options  and  potential 
outcomes  of  a  particular  decision.  This  taxonomy  of  repercussions  provides  a  visual  "map"  that 
allows  a  decision  maker  to  identify  and  choose  the  most  strategic  option.  This  structure  also 
helps  organize  the  overwhelming  amounts  of  information  that  are  relevant  to  a  decision  maker's 
specific  topic.” 


146  Anonymous  pp.  17-18. 

147  Anonymous  pp.  19-20. 

14S  Anonymous  pp.  27-29. 

149  Anonymous  pp.  31-33. 

150  Cressley,  C.A.  "Predictive  Markets"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  & 
D.E.  Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  95-102. 

151  Annibale,  P.  "Decision  Trees"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  &  D.E. 
Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  71-79. 
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Scenario  Planning1*2 

Description  -  “Scenario  planning  derives  from  the  observation  that,  given  the  impossibility  of 
knowing  precisely  how  the  future  will  play  out,  a  good  decision  or  strategy  to  follow  is  one  that 
plays  out  well  across  several  possible  futures.  The  purpose  of  scenario  planning  is  not  to  pinpoint 
future  events  but  to  highlight  large-scale  forces  that  push  the  future  in  different  directions.  It's 
about  making  these  forces  visible,  so  that  if  they  do  happen,  the  planner  will  at  least  recognize 
them.” 

Futures-Based  Analytical  Approach 153 

Description  -  “A  futures-based  analytical  approach  starts  with  an  assumed  future  condition  and 
works  backward  to  identify  relevant  current  trends  and  an  "unknown  space"  that  links  those 
trends  with  the  assumed  future.  The  "unknown  space"  is  the  missing  link,  or  trigger,  that  propels 
the  current  state  of  affairs  into  the  assumed  future. . .  A  futures-based  analytical  approach  is  a  tool 
to  identify  the  key  drivers  of  future  catastrophic  conditions  in  order  to  develop  countenneasures 
that  prevent  the  very  act,  or  state  of  affairs,  that  is  assumed  to  be  inevitable.” 

Delphi  Method154 

Description  -  “. .  .involves  the  collection  of  views  and  opinions  on  a  given  problem  from  a 
number  of  "experts"  and  commentators,  providing  their  input  separately  to  the  central  point.  At 
the  completion  of  that  round  of  gathering  their  views,  the  individual  group  members  are  provided 
with  feedback  about  the  group's  overall  judgments,  and  a  second  round  of  opinions  is  called  for 
on  the  same  problem.  This  process  continues  to  narrow  the  estimates  until  the  tasking  unit  is 
satisfied  that  some  form  of  agreement  has  been  reached — an  agreement  gained  through  sharing 
of  individual  expert  opinions  based  on  differing  levels  and  types  of  experience,  all  brought  to 
bear  on  a  common  problem.” 


152  Hippier,  R.  "Scenario  Planning"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E.  Mosco  &  D.E. 
Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  54-62. 

153  Kincaid,  B.  "Futures-based  Analytical  Approach"  in  The  Analyst's  Cookbook.  Volume  I.,  eds.  K.J.  Wheaton,  E.E. 
Mosco  &  D.E.  Chido,  Mercyhurst  College  Institute  of  Intelligence  Studies  Press,  Erie,  PA,  2006,  pp.  44-53. 

154  McDowell,  D.  Strategic  Intelligence:  A  Handbook  for  Practitioners,  Managers  and  Users,  Istana  Enterprises  Pty. 
Ltd,  CoomaN.S.W,  1998,  pp.  142. 
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9.0 


ASSIGNMENT  OF  MEANING 


...why  under  certain  circumstances  human  beings  by  themselves  can  come  to  know  things 
that  are  hidden  while  these  things  remain  hidden;  that  is,  why  they  can  know  even  when 
they  have  no  direct  access  to  these  things  and  those  who  have  would  not  talk.  (Lai 

2005)m 

9.1  Before  We  Start 

In  the  “Introduction  to  the  FAC”  section,  we  defined  the  core  objective  of  intelligence  analysis  as 
discerning  what  had  been  previously  unknown:  the  hidden  or  classified  plans,  capabilities,  and 
intentions  of  a  foreign  nation  or  adversary.  While  this  has  been  said  before,  it  cannot  be  said  too 
many  times  or  too  often.  The  challenge  of  needing  to  know  what  others  do  not  want  you  to 
discover  is  central  to  seeking  information  and  developing  and  using  the  analytical  approaches 
that  we  discuss  in  this  section. 

The  objective  of  learning  the  unknown  applies  to  any  intelligence  issue  you  address.  It  also 
establishes  a  measure  for  the  success  of  your  analytical  results.  That  is,  did  you  discover 
something  that  was  previously  unknown?  If  your  analysis  does  not  add  anything  to  the 
information  upon  which  you  based  your  analysis,  you  are  not  providing  the  assessment  needed 
by  those  you  support. 

But  how  is  one  to  know  what  is  unknown?  While  some  might  say  that  discovering  the  unknown 
is  not  possible,  our  life  experiences  tell  us  that  it  is.  Detectives  solve  crimes  and  scientists 
discover  many  previously  unknown  features  of  the  natural  world.  So,  how  is  one  to  accomplish 
knowing  the  previously  unknown?  In  the  Introduction  and  Background  section,  we  described 
two  general  analysis  processes  as  defined  by  Heuer  and  Kent.  In  the  Analysis  Techniques 
section,  we  reviewed  various  SATs.  However,  processes  and  techniques  alone  are  not  sufficient 
to  aid  you  in  finding  the  unknown.  This  FAC  section  and  the  one  that  follows  it,  Assessment, 
provide  a  more  complete  description  of  an  approach  that  helps  you  discover  the  unknown. 

The  first  step  in  finding  the  unknown  is  to  assign  meaning  to  each  of  the  individual  pieces  of 
relevant  information  that  resulted  from  your  information  acquisition  efforts.  However,  before  we 
can  start  with  our  explanation  of  an  approach  to  assigning  meaning,  we  need  to  define  terms.  The 
terms  we’re  defining  are  used  frequently  in  the  intelligence  analysis  literature,  but  not  necessarily 
with  the  same  meaning  or  definition.  In  addition,  the  tenns  are  used  frequently  in  everyday 
conversations,  with  meanings  different  from  our  usage  in  the  FAC. 

The  following  definitions  are  those  that  we  use  in  the  FAC  to  discuss  approaches  for  the 
assignment  of  meaning,  as  well  as  for  subsequent  FAC  sections.156 

Fact  -  repeatable  measurements  of  various  objects  or  phenomena.  The  most  distinctive 
feature  of  a  fact  is  that  it  represents  a  value  which  involves  little  or  no  human 
interpretation.  The  tenn  fact  is  sometimes  used  synonymously  with  truth  or  reality,  as 
distinguishable  from  conclusions  or  opinions. 


155  Lai,  T.  The  Art  of  Detection,  2005.  Available:  http://plato.ucs.mun.ca/~tlai/Art/ArtTitle.htm. 

156The  sources  used  to  form  these  definitions  include  the  Merrill-Webster  Open  Dictionary,  Wikipedia,  and 
Wiktionary.  However,  the  definitions  given  do  not  conform  exactly  with  the  definitions  from  those  sources  since  the 
definitions  were  tailored  to  relate  to  the  cognition  of  analysis. 
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Data  -  output  by  a  sensing  device  or  organ  that  includes  both  useful  and  irrelevant  or 
redundant  content.  Data  must  be  processed  into  a  form  understandable  by  humans 
without  adding  additional  information  to  the  content.  Data  may  or  may  not  be  based  on 
repeatable  measurements.  157 

Information  -  content  resulting  from  physical  or  cognitive  actions.  For  the  purposes  of 
the  FAC,  we  use  the  word  information  for  content  that  is  communicated  from  one  person 
to  another.158 

Knowledge/Beliefs  -  In  the  FAC,  we  consider  knowledge  and  beliefs  to  be  the 
interpretation  of  facts,  data,  and  infonnation.  From  a  cognitive  standpoint,  we  make  no 
distinction  between  knowledge  and  beliefs. 

Defining  knowledge  and  beliefs  is  like  being  in  a  tar  pit.  People  have  been  arguing  over 
the  meaning  and  difference  between  these  two  words  since  the  time  of  the  Greek 
philosophers,  maybe  even  earlier,  but  written  records  of  the  arguments  haven’t  survived. 
Differences  in  the  definition  of  each  word  have  been  the  grist  for  innumerable 
considerations  ever  since.  We  don’t  solve  this  problem  in  the  FAC,  but  for  our  purposes, 
the  two  words  are  cognitively  the  same.  Both  represent  a  degree  of  conviction  about  the 
truth  of  a  statement  or  the  reality  of  some  being  or  phenomenon,  especially  when  based 
on  examination  of  information.  If  a  difference  does  exist  between  knowledge  and  beliefs 
it  depends  on  the  amount  of  infonnation  used  to  support  one  or  the  other  and  the 
interpretation  of  how  conclusive  that  supportive  information  is.  We  know  in  some  cases 
that  people  assimilate  or  learn  what  others  consider  to  be  knowledge,  but  their  beliefs 
remain  unchanged.159  As  we  pointed  out  in  the  Analysts  as  Individuals  section,  some 
beliefs  exist  without  any  information  to  support  them. 

Clue  -  facts,  data,  or  information  that  suggests  an  explanation  for  the  information 
relevant  to  a  topic  about  an  unknown.  Clues  reveal  something  about  a  hidden  aspect  of 
structure,  habits,  or  intent. 

Evidence  -  evidence  is  relevant  to  a  given  hypothesis  if  the  evidence  either  increases  or 
decreases  the  likeliness  of  the  hypothesis.  Some  usages  of  the  word  suggest  that  all  facts, 
data,  and  information  relevant  to  an  intelligence  issue  are  evidence.  However,  others, 
most  notably  Schum,  define  evidence  as  content  relevant  to  the  hypotheses  being 
considered.160  We  use  Schum’s  definition  in  the  FAC. 

Assumptions  -  statements  or  descriptions  that  are  treated  as  if  they  are  true  without  the 
necessity  of  proving  them  true.  Assumptions  are  often  used  to  establish  boundary 
conditions  for  subsequent  analysis.  For  example,  the  substantive  areas  identified  in  the 


157  Both  Kent  and  Heuer  use  the  term  data  to  describe  the  input  to  analysis.  Their  usage  is  not  consistent  with  the 
FAC  definition.  Their  intent  is  best  expressed  by  using  the  word  “information.” 

I5S  The  word  knowledge  is  also  frequently  used  to  describe  the  content  being  communicated  between  persons.  While 
the  content  being  communicated  may  be  knowledge  to  the  individual  doing  the  communicating,  the  content  may  not 
be  knowledge  to  the  persons  receiving  the  content  for  reasons  we’ll  explain  later. 

159  For  example,  see  Alexander,  P.A.  &  Dochy,  F.  J.  R.  C.  "Conceptions  of  Knowledge  and  Beliefs:  A  Comparison 
across  Varying  Cultural  and  Educational  Communities,"  American  Educational  Research  Journal,  Vol.  32,  No.  2, 
1995,  pp.  413-442. 

160  Flughes,  F.  J.  &  Schum,  D.A.  The  Art  and  Science  of  the  Process  of  Intelligence  Analysis,  Joint  Military 
Intelligence  College,  Washington,  D.C.,  2003,  p.  82. 

154 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 

88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


analytic  plan  are  assumptions  about  what  is  important  to  the  intelligence  issue.  On 
occasion,  an  assessment  may  label  assumptions  those  statements  that  actually  represent 
the  results  of  analysis.  For  example,  statements  regarding  another  country’s  intentions  or 
capabilities,  governmental  processes,  and  relative  strength  of  political  forces  may  be 
labeled  assumptions.  However,  these  statements  are  clearly  judgments  that  should  be  the 

result  of  intelligence  analysis.161  Such  statements  are  not  assumptions  except  when  used 

1 62 

to  establish  boundary  conditions  in  a  “what  if’  analysis. 

Hypothesis  -  a  tentative  conjecture  explaining  an  observation,  phenomenon,  or  scientific 
problem  that  can  be  tested  by  further  observation,  investigation,  or  experimentation.  A 
hypothesis  can  be  established  whenever  an  explanation  is  desired  for  a  collection  of 
information.  While  both  assumptions  and  hypotheses  have  a  common  element  of 
“assumed  to  be  true”,  they  differ  in  their  intent.  An  assumption  is  a  boundary  condition 
for  analysis.  A  hypothesis  is  an  explanation  of  what  results  from  the  analysis.  In  the 
context  of  mindware,  a  hypothesis  is  a  tentative  belief  that  an  analyst  uses  to  evaluate 
information. 

Opinion/Judgment  -  these  two  words  are  used  to  define  one  another.  An  opinion  is  a 
judgment  and  a  judgment  is  an  opinion.  In  both  cases,  they  represent  the  transition  from  a 
hypothesis,  or  tentative  explanation,  to  an  accepted  explanation.  In  typical  intelligence 
usage,  the  only  distinction  between  the  words  opinion  and  judgment  is  the  degree  of 
authority  normally  associated  with  the  word.  A  judgment  is  considered  more  authoritative 
than  an  opinion.  In  either  case,  the  development  of  an  opinion  or  a  judgment  provided  to 
an  external  audience  is  the  final  stage  of  analysis.  Opinions  and  judgments  represent  the 
output  of  the  entire  cognitive  process  of  analysis. 

9.2  Now  Can  We  Begin  To  Do  Analysis? 

No,  not  yet.  Certain  concepts  are  the  foundation  for  the  analysis  approaches  embedded  in  FAC. 
While  we  discussed  most  of  these  concepts  in  earlier  sections  of  the  FAC,  we  provide  a 
consolidated  review  in  this  section  to  reinforce  the  concepts  before  you  engage  in  analysis. 

9.2.1.  First  Concept:  Originator’s  Meaning 

The  first  concept  to  review  is  that  the  recipient  of  infonnation  determines  the  meaning  of  the 
information.  In  view  of  the  definitions  above,  we  broaden  this  statement  to  say  that  the  recipient 
determines  the  meaning  of  all  input  -  facts,  data,  and  information.  The  idea  that  input  has  a 
single  meaning  and  is  interpreted  in  one  way,  by  all  its  recipients,  has  been  proven  incorrect.  As 
we  pointed  out  in  the  Introduction  and  Background  section,  the  results  of  extensive  scientific 
research  support  the  concept  that  the  recipient  detennines  the  meaning  of  all  input.  The 
intelligence  literature  also  shows  wide  awareness  of  this  concept  even  though  the  ramifications 
of  the  concept  do  not  seem  to  be  fully  appreciated. 


161  Heuer,  R.J.  &  Pherson,  R.H.  Structured  Analytic  Techniques  for  Intelligence  Analysis,  CQ  Press,  Washington, 
D.C.,  2011,  p.  31. 

162  In  the  intelligence  literature,  the  term  assumption  is  frequently  misused  as  synonymous  with  the  term  belief. 

163  This  definition  is  similar  to  the  one  used  by  Schum.  See:  Schum,  D.A.  Evidence  and  Inference  for  the 
Intelligence  Analyst.  Volume  1,  University  Press  of  America,  Lanham,  MD,  1987. 
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We  now  know  or  believe  that  variances  in  meaning  are  the  result  of  the  variances  in  mindware 
between  individuals.164  In  view  of  the  definitions  we  discussed  above,  we  define  mindware  as 
content  (facts,  data,  and  information),  knowledge/beliefs,  and  goals.  As  we  discussed  in  previous 
sections  of  the  FAC,  an  individual’s  mindware  does  not  remain  constant.  Changes  can  and  do 
occur  to  mindware  frequently.  As  a  result,  the  meaning  of  information  is  always  susceptible  to 
change. 

While  the  statement  that  information  does  not  have  a  single  meaning  is  true,  we  attach  one  very 
important  qualifier  to  the  statement.  The  statement  is  valid  in  terms  of  the  recipients  of  the 
information;  however,  the  originator  of  the  information  undoubtedly  and  most  assuredly  intends 
to  communicate  a  single  meaning.  For  intelligence  analysis,  there  are  many  people  that  are  part 
of  your  target  of  study  that  originate  information  (e.g.,  foreign  decision  makers,  weapons 
designers,  military  planners,  etc).  Understanding  the  intended  meaning  of  these  originators  is  key 
to  you  understanding  the  target.  This  distinction  between  meaning  to  recipients  and  meaning  to 
the  originator  is  what  separates  intelligence  analysis  from  most  other  forms  of  analysis.  In  most 
forms  of  analysis,  the  objective  is  to  assign  meaning  and  determine  what  the  information  means 
to  you.  In  intelligence  analysis,  the  objective  is  to  assign  meaning  to  detennine  what  the 
originator  meant. 

As  a  consequence,  one  of  the  major  objectives  of  intelligence  analysis,  that  is,  acquiring 
information  and  analyzing  it,  is  to  establish  the  mindware  of  the  originator.  Thus,  your  analysis 
has  two  components.  The  first  component  is  to  establish  the  context  of  an  individual  piece  of 
information  in  order  to  build  your  alternate  mindware.  The  second  component,  done  only  after 
you  establish  the  context  of  information,  is  to  begin  the  assignment  of  meaning  as  it  was 
intended  by  the  originator. 

Your  success  at  adopting  these  various  perspectives  in  order  to  assign  meaning  to  information 
depends  on  the  state  of  your  epistemological  beliefs.  If  you’ll  recall  from  the  Analysis  Techniques 
section  of  the  FAC,  epistemological  beliefs  range  from  naive  to  sophisticated  and  describe  how  an 
individual  views  knowledge  itself.  A  naive  epistemological  belief  views  knowledge  as  absolute, 
organized  in  isolated  bits  and  pieces,  handed  down  by  authority,  and  that  a  person’s  ability  to 
learn  is  fixed  at  birth.  More  sophisticated  epistemological  beliefs  are  characterized  by  views  that 
include  the  perspective  that  knowledge  is  tentative,  organized  as  interwoven  concepts,  derived 
through  reason,  and  that  a  person’s  ability  to  learn  can  change  throughout  a  lifetime.  In  order  to 
accomplish  the  work  we  describe  in  this  section  of  the  FAC,  we  rely  on  more  sophisticated 
epistemological  beliefs. 

The  second  concept  we’ll  review  relates  to  developing  the  alternate  mindware  of  the  originator. 

9.2.2.  Second  Concept:  Habits 

The  second  basic  concept  we’ll  review  in  this  section  is  habits,  or  the  predictability  of  people. 
Some  authors  think  that  discerning  predictability  is  an  impossible  task: 

But  even  the  best-functioning  intelligence  service  cannot  be  accounted  upon  reliably  to 
predict  the  actions  of  foreign  powers:  divining  political  intentions  is  far  and  away  the 
most  difficult  aspect  of  intelligence  work.  This  holds  especially  true  of  dictatorial 
regimes,  with  which  U.S.  intelligence  is  particularly  concerned,  because  their  decisions 


164  As  a  reminder,  mindware  consists  of  knowledge,  beliefs  and  goals  (as  defined  by  Stanovich  and  discussed  in  the 
“Analysts  as  Individuals”  and  “Thinking  about  Thinking”  sections). 
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are  in  the  hands  of  unstable  and  impulsive  individuals  subject  to  few  if  any  external 
controls.  It  is  hard  to  predict  the  behavior  of  unpredictable  personalities.165 

However,  the  literature  we  reviewed  to  develop  the  FAC  reflects  a  growing  belief  that 
irrationality  is  not  the  same  as  unpredictability.  The  best  single  example  of  this  belief  is 
manifested  in  the  title  of  a  recent  book,  Predictably  Irrational. 166  Other  publications  we 
reviewed  also  characterized  people  as  predictable  since  they  are  “creatures  of  habit”. 

It  can  be  difficult  to  make  day-to-day  predictions  of  an  irrational  leader's  behavior,  but  in 
time  a  general  pattern  of  behavior  will  gradually  emerge  thereby  helping  the  observer  to 
gauge  some  of  the  leader's  reactions  and  readiness  to  take  risks,  if  not  to  make  more 
precise  forecasts.167 

Anticipating  behavior  patterns  can  be  done  at  the  individual  and  group  level.  People  are 
predictable  and  creatures  of  habit  if  you  are  patient  and  skilled  enough  to  locate  the  cues. 
There  are  a  multitude  of  possible  places  to  look  for  the  puzzle  pieces,  usually  in  a  data 
set.168 

We  are  all  creatures  of  habit,  and  nowhere  is  this  more  evident  than  in  the  way  we  make 
decisions.  If  something  worked  well  for  us  once,  we  are  likely  to  repeat  that  action.  It  is 
therefore  important  to  examine  the  history  of  a  manager  s  decisions.  This  is  among  the 
most  important  and  easiest  of  the  profiling  dimensions  to  discern,  because  it  is  history 
and  others  know  about  it.169 

One  might  question  this  heavy  emphasis  on  knowing  and  understanding  the  past  given  that 
almost  all  intelligence  issues  relate  to  future  events.  Winston  Churchill  provided  an  answer  to 
such  a  question: 

The  further  back  you  look  the  further  ahead  you  can  see.170 

Drucker  also  emphasized  the  role  of  the  past  in  predicting  the  future: 

In  human  affairs  political,  social,  economic,  and  business  it  is  pointless  to  try  to  predict 
the  future,  let  alone  attempt  to  look  ahead  75  years.  But  it  is  possible  and  fruitful  to 
identify  major  events  that  have  already  happened,  irrevocably,  and  that  therefore  will 
have  predictable  effects  in  the  next  decade  or  two.  It  is  possible,  in  other  words,  to 
identify  and  prepare  for  the  future  that  has  already  happened.171 

The  FAC  embraces  the  concept  that  you  can  predict  a  high  percentage  of  someone’s  future  acts  if 
you  work  to  understand  that  person’s  past  physical,  cognitive,  and  emotional  habits.  The  need  to 
understand  the  past  does  not  imply  that  you  can  predict  future  behavior  by  simply  doing  a  linear 


1  Pipes,  R.  "What  to  do  about  the  CIA,"  Commentary,  Vol.  99,  No.  3,  1995,  p.  3. 

166.  Ariely,  D.  Predictably  Irrational:  The  Hidden  Forces  that  Shape  our  Decisions,  Harper,  New  York,  2008. 

167  Handel,  M.I.  "Intelligence  and  the  Problem  of  Strategic  Surprise,"  in  Paradoxes  of  Strategic  Intelligence:  Essays 
in  Honor  of  Michael  I.  Handel,  eds.  R.K.  Betts  &  T.G.  Mahnken,  Frank  Cass,  London,  UK,  2003,  p.  28. 

16S  Ronczkowski,  M.  Terrorism  and  Organized  Hate  Crime:  Intelligence  Gathering,  Analysis,  and  Investigations, 
2nd  edn,  CRC  Press,  Boca  Raton,  FL,  2006,  p.  77. 

169  Fuld,  L.M.  The  New  Competitor  Intelligence:  The  Complete  Resource  for  Finding,  Analyzing,  and  using 
Information  about  Your  Competitors,  J.  Wiley,  New  York,  NY,  1995,  p.  406. 

170  Goodman,  M.S.  &  Omand,  D.  "Teaching  Intelligence  Analysts  in  the  UK:  What  Analysts  Need  to  Understand: 
The  King's  Intelligence  Studies  Program,"  Studies  in  Intelligence,  Vol.  52,  No.  4,  2008,  pp.  1-12. 

171  Drucker,  P.  "The  Future  that  has  Already  Happened,"  The  Futurist,  Vol.  32,  No.  8,  1998,  p.  1. 
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extrapolation  of  past  events  into  the  future.  The  physical,  cognitive,  and  emotional  habits  of  an 
individual  occur  in  a  given  context.  Understanding  that  context  is  key  to  understanding  the 
applicability  of  past  events  to  the  future.  If  a  person’s  context  has  changed  from  what  it  was  in 
the  past,  the  change  in  context  may  affect  the  habits  under  study.  This  is  why  it  is  so  important  to 
understand  both  habits  and  the  context  in  which  they  occur.  Analytically,  it  is  important  to  be 
aware  of  when  a  person’s  context  has  changed  or  to  identify  what  is  unknown  about  the  new 
context.  In  subsequent  parts  of  this  section,  we  will  describe  how  to  use  extensive  information 
acquisition  and  the  assignment  of  meaning  to  information  in  order  to  determine  the  why  of  a 
person’s  actions.  The  goal  of  this  effort  is  to  detennine  the  habits  of  people,  the  context  in  which 
those  habits  take  place,  and  the  behavior  that  will  likely  result. 

9.2.3.  Third  Concept:  Structure 

The  FAC  is  based  on  the  concept  that  almost  all  aspects  of  physical  and  cognitive  acts  take  place 
within  a  structure.  If  you  are  considering  an  individual,  some  relationship  exists  between  one  act 
and  the  acts  which  preceded  it.  For  example,  consider  offender  profiling.  Law  enforcement  may 
use  a  profile  developed  by  the  FBI  to  search  for  suspects  for  a  particular  crime.  The  offender 
profile  used  is  based  on  the  choices  made  by  the  criminal  before,  during,  and  after  a  crime  and 
other  information  gathered  by  law  enforcement.  These  choices  form  a  structure  that  defines  the 
offender’s  behavior;  this  structure  can  be  perceived  from  one  instance  of  the  offender’s  behavior 
to  another  and  provides  a  useful  tool  for  identifying  the  person  who  committed  the  crime. 

An  offender  profile  is  one  kind  of  structure.  Structures  are  also  evident  in  decision  making, 
languages,  machinery,  communication,  crime,  atoms,  solar  systems  and  molecules. 

If  you  are  attempting  to  determine  the  originator’s  meaning,  you  need  to  understand  the 
originator’s  structure.  Clues  provide  the  means  for  discovering  the  originator  of  information’s 
structure,  just  as  clues  provide  the  insight  needed  for  developing  an  offender  profile. 

•  Clues  are  the  characteristics  of  structures.  The  more  characteristics  of  a  structure  we 
know,  the  easier  it  is  to  distinguish  it  from  other  structures.172 

1  7T 

•  Clues  are  the  characteristics  of  structures,  disguised. 

•  When  something  hidden  is  known,  therefore,  it  is  known  because  it  has  a  structure 
(or  is  a  part  of  some  structure).  If  it  has  a  structure  (or  is  a  part  of  some  structure) 
clues  can  lead  us  to  it.  Knowledge  of  the  hidden,  therefore,  depends  on  structure. 

No  structure,  no  clues.  No  clues,  no  knowledge.174 

9.2.4.  Summary  -  Three  Concepts 

Originator’s  meaning,  habits,  and  structure  form  the  conceptual  foundation  for  the  FAC: 

•  The  recipient  of  information  determines  the  infonnation’s  meaning.  However,  a 
distinctive  characteristic  of  intelligence  analysis  is  that  an  analyst  must  detennine 
what  the  originator  meant  by  the  infonnation.  This  requires  that  the  analyst  re¬ 
create  the  mindware  of  the  originator  during  intelligence  analysis. 


172  Lai  [unpaged]. 

173  Lai  [unpaged]. 

174  Lai  [unpaged]. 
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•  A  characteristic,  and  perhaps  unique,  structure  exists  which  determines  the  actions 
of  people  and  organizations. 

•  People  are  creatures  of  habit,  and  habits  are  highly  influenced  by  the  structure  in 
which  they  occur.  Understanding  both  a  person’s  structure  and  his  or  her  physical, 
cognitive,  and  emotional  habits  can  provide  a  reliable  basis  for  prediction  of  future 
events. 

•  Information  may  exist  that  will  reveal  the  structure,  habits,  and  intent  of  the 
originators  of  the  information. 

•  Clues  are  the  means  used  to  reveal  the  hidden  aspects,  structures,  habits,  and 
intentions  of  the  subject  of  a  study.  The  ability  to  identify  these  clues  is  dependent 
on  an  analyst’s  ability  to: 

>  Find  all  the  information  that  can  lead  to  a  recognition  of  clues  and; 

>  Apply  mindware  to  this  infonnation. 

9.3  And  the  Consequences  Are? 

If  you  agree  with  the  three  concepts  we  discussed  above,  what  may  not  yet  be  evident  is  how  you 
incorporate  these  concepts  into  your  analytical  practices.  To  illustrate,  let’s  return  to  our 
statement  about  the  essence  of  intelligence  analysis:  assigning  meaning  to  information.  In  the 
Introduction  and  Background  section,  we  said  that  attributing  meaning  to  information  is  based  on 
the  answers  to  two  questions:  what  and  why  ?  The  what  question  can  usually  be  answered  by  the 
substance  found  in  the  relevant  infonnation  you  retrieve.  However,  the  relevant  information  you 
retrieved  rarely  provides  explicit  infonnation  on  the  why  question.  The  relevant  information  that 
you  retrieved  during  information  acquisition  may  have  additional  information  about  people, 
places  and  things.  Researching  people,  places,  and  things  beyond  the  intelligence  issue  itself 
often  contributes  to  understanding  why  something  happened. 

How  can  I  use  people,  places,  and  things  to  answer  the  why  question  for  the  intelligence  issue? 

If  you  consider  habits,  the  second  basic  concept  of  the  FAC,  the  activity  reflected  in  the  relevant 
information  is  likely  to  be  consistent  with  the  prior  activities  of  the  people  involved  with  the 
intelligence  issue.  The  places  mentioned  in  the  relevant  infonnation,  either  directly  or  by  the 
involvement  of  people  associated  with  the  places,  are  likely  to  be  consistent  with  the  mission  and 
past  activities  of  the  places.  If  things  are  involved,  the  purpose  for  their  involvement  is  likely  to 
be  consistent  with  past  usage  and  can  generate  additional  clues. 

Important  information  about  people,  places,  and  things  that  answers  the  why  question  is  unlikely 
to  be  in  any  one  document.  However,  it  can  be  obtained  from  an  analysis  of  all  the  information 
on  the  people,  places,  and  things  directly  associated  with  the  relevant  infonnation  on  associated 
people,  places  and  things.  The  flow  associated  with  the  acquisition  of  all  this  information  is 
shown  in  Figure  6. 
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Figure  1:  General  Flow  of  Information  Acquisition  and  Analysis 
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Figure  6:  Information  Acquisition  Flow 

We  described  the  first  step  of  this  approach  in  the  “Information  Acquisition”  section.  The  result 
of  the  first  step  of  information  acquisition  is  a  collection  of  relevant  information  and  a  listing  of 
people,  places  and  things  associated  with  the  relevant  information.  The  listing  of  people,  places, 
and  things  is  the  input  for  the  search  topics  in  the  next  step  in  Figure  6.  You  review  the  results 
of  searching  in  the  next  step  to  identify  the  information  about  person,  places,  and  things 
and  record  a  new  list  of  people,  places,  and  things  to  be  searched  further.  These  newly 
identified  people,  places  and  things  will  be  the  search  topics  for  the  next  step. 

As  you  continue  to  search  and  identify  new  people,  places  and  things,  you  acquire  more  and 
more  information  about  the  context  of  the  information  for  which  you  plan  to  assign  meaning. 
Ideally,  you  repeat  this  process  until  you  can  no  longer  identify  any  new  people,  places  or  things. 
At  this  point  you  have  acquired  all  the  contextual  infonnation  that  relates  to  infonnation  to 
which  you  hope  to  assign  meaning.  However,  your  searching  effort  may  stop  before  you  reach 
the  ideal  state  -  either  because  of  limitations  in  time  or  because  the  content  of  the  resultant 
searches  seems  too  far  removed  from  the  intelligence  issue  driving  the  analysis. 

After  you  acquire  all  the  information  described  in  Figure  6,  your  analysis  can  start  to  answer  the 
why  question  for  the  people,  places  and  things  you  identified.  However,  the  concentration  on 
answering  why  does  not  imply  ignoring  the  substantive  content  contained  in  the  information 
associated  with  people,  places,  and  things.  Even  though  the  substantive  content  may  not  be 
relevant  to  the  intelligence  issue,  you  must  consider  the  substantive  content  of  the  information 
relevant  to  the  people,  places,  and  things  since  it  can  be  a  valuable  input  to  understand  the  why. 
In  addition,  when  you  search  on  people,  places,  and  things  you  may  retrieve  more  information 
relevant  to  the  intelligence  issue  that  you  hadn’t  anticipated  when  you  developed  the  search 
topics. 
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We’ll  discuss  the  approach  for  answering  the  why  question  later  in  this  section.  However,  there  is 
something  that  is  important  to  recognize  that  may  not  be  immediately  obvious.  It  is  that  a  new 
purpose  or  goal  is  established  for  answering  the  why  question  for  each  of  the  people,  places  and 
things.  It  is  also  important  to  recognize  that  answering  each  why  question  represents  a  complete 
analysis  effort  as  defined  in  FAC.  To  answer  each  why  question,  your  purpose  or  goal  is  to 
answer  that  why  question,  including  knowing  how  you  are  going  to  accomplish  the  analysis 
required,  detennining  what  infonnation  you  will  seek  and  from  where,  acquiring  that 
information,  assigning  meaning,  marshalling  evidence,  generating  tentative  hypotheses,  and 
forming  an  explanation  or  explanations  that  are  well  supported  by  the  evidence.  As  we’ll  discuss 
later,  when  an  explanation  is  well  supported  by  the  evidence,  it  becomes  a  belief.175  It  is  a 
general  principle  of  the  FAC  that  the  cognition  associated  with  any  question  related  to  addressing 
the  intelligence  issue  is  the  same  as  that  associated  with  directly  addressing  the  intelligence 
issue. 

As  you  undoubtedly  have  noticed,  we  are  saying  that  before  assigning  meaning  to  the 
information  relevant  to  the  intelligence  issue,  a  considerable  amount  of  assigning  meaning, 
marshalling  evidence,  and  generating  hypotheses  has  to  be  done  on  the  information  related  to 
the  relevant  information.  This  section  provides  you  with  overall  information  on  marshalling 
evidence  and  tentative  hypotheses  generation  for  the  individual  pieces  of  infonnation  you 
retrieve  during  this  process. 

In  the  next  FAC  section,  we’ll  provide  more  information  about  the  analysis  directly  associated 
with  addressing  your  intelligence  issue.  This  will  include  information  about  marshalling 
evidence  and  the  beliefs  that  result  from  your  tentative  hypotheses  about  the  individual  pieces  of 
information.  We  will  also  include  information  about  generating  additional  tentative  hypotheses 
for  the  aggregation  of  the  beliefs  resulting  from  your  analysis  of  the  information.  Ultimately, 
your  overall  belief(s)  will  form  the  basis  for  the  judgment(s)  that  will  be  communicated  to  an 
external  audience  to  address  the  intelligence  issue. 

We  can  now  add  the  following  to  the  previous  summary  of  the  FAC  basic  conceptual  foundation: 

•  The  meaning  of  information  is  dependent  on  context.  Since  each  individual 
provides  the  context,  the  meaning  of  the  infonnation  is  likely  to  be  as  varied  as  the 
differences  among  individuals. 

•  The  challenge  for  the  analyst  is  to  create  the  originator’s  context  for  the  information 
obtained. 

•  Analysis  to  address  an  intelligence  issue  consists  of  many  layers  of  analysis,  but 
cognitively  the  nature  of  the  layers  is  the  same.  Overall,  the  assignment  of  meaning 
proceeds  from  the  information  least  directly  related  to  the  intelligence  issue  to  that 
most  directly  related  (i.e.,  from  the  “bottom  up”  on  Figure  6 — more  on  this  later). 

9.4  Now  Are  We  Ready  To  Start  With  Analysis? 

Y es,  we  are  finally  ready.  Analysis  is  a  highly  iterative  and  cognitive  process  with  elements  of 
creativity  since  you  must  generate  new  thoughts.  Consequently,  it  can’t  be  done  using  a  checklist 
or  a  fixed  set  of  steps.  However,  from  the  preceding  material  you  can  sense  a  more  effective 

175  As  you  will  see  in  the  descriptions  that  follow,  at  any  point  in  the  FAC  approach  for  assigning  meaning  (i.e., 
acquiring  context,  answering  the  what  question,  and  answering  the  why  question),  you  may  be  operating  with  one  or 
more  hypotheses,  explanations,  or  beliefs. 
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sequence  for  doing  analysis,  such  as  when  certain  infonnation  should  be  considered  as  well  as 
the  type  of  information  to  be  considered.  As  we  described  in  the  “Analysis  Techniques”  section 
(e.g.,  regarding  thinking  dispositions  and  cognitive  skills),  it  is  important  to  use  an  orderly, 
focused  approach.  What  follows  is  designed  to  help  you  do  your  analysis  in  an  orderly,  focused 
way,  recognizing  that  the  analysis  is  at  times  not  sequential  and  is  quite  unstructured  in  nature. 
The  approaches  will  guide  you  on  the  sequence  for  analysis  and  the  infonnation  that  you 
consider  as  you  build  your  knowledge  base,  develop  your  understanding  of  the  current  status  of 
the  activities  and  intentions  associated  with  your  intelligence  issue,  and  do  your  original 
thinking.  This  approach  is  designed  to  guide  your  cognition,  not  limit  it,  as  you  are  engaged  in 
the  essential  elements  of  analysis,  i.e.,  assigning  meaning,  marshalling  evidence,  forming 
tentative  hypotheses,  and  developing  beliefs. 

9.4.1,  Approach  for  Assigning  Meaning  and  Analysis 

As  was  mentioned  in  the  “Introduction  to  the  FAC”  section,  the  goal  of  assigning  meaning  to 
information  is  to  detennine  the  implications  of  hidden  or  special  significance  from  the  content  of 
the  information.  When  you’re  assigning  meaning  using  the  approach  in  this  FAC  section,  each 
piece  of  infonnation  is  essentially  treated  equally.  You  will  consider  each  piece  of  information 
and  derive  its  meaning.  As  described  in  the  “Introduction  to  the  FAC”  section,  information  is 
valuable  if  it  changes  your  understanding.  Determining  the  value  of  the  information  occurs  as  the 
information  is  related  to  the  tentative  hypotheses  that  you  generate.  As  we  discussed  earlier, 
when  a  tentative  hypothesis  is  well  supported  by  information,  it  becomes  a  belief.  Consequently, 
the  value  of  infonnation  is  based  on  the  degree  to  which  it  changes  your  understanding  and 
supports  or  refutes  your  hypotheses. 

The  meaning  you  derive  from  information  answers  two  questions:  what  was  done  and  why  it  was 
it  done.  Rarely  will  a  single  document  contain  the  infonnation  needed  to  answer  these  questions, 
which  is  why  we  refer  to  it  as  hidden.  The  fact  that  some  information  is  hidden  is  the  basis  for 
the  common  assertion  that  information  doesn’t  speak  for  itself.  But,  as  Schum  states,  better 
questions  allow  you  to  “’’hear”  what  it  [the  information]  has  to  say.”  Thus,  as  you  transition  to 
the  assignment  of  meaning  phase  of  analysis,  your  primary  mode  of  thinking  is  to  be  inquisitive. 
The  importance  of  being  inquisitive  has  been  best  said  as: 

It  is  not  possible  to  be  a  good  thinker  and  a  poor  questioner. 

Questions  define  tasks,  express  problems,  and  delineate  issues.  They  drive  thinking 
forward.  Answers,  on  the  other  hand,  often  signal  a  full  stop  in  thought.  Only  when  an 
answer  generates  further  questions  does  thought  continue  as  inquiry.  A  mind  with  no 
questions  is  a  mind  that  is  not  intellectually  alive.  No  questions  (asked)  equals  no 
understanding  (achieved).  Superficial  questions  equal  superficial  understanding,  unclear 
questions  equal  unclear  understanding.  If  your  mind  is  not  actively  generating  questions, 
you  are  not  engaged  in  substantive  learning.177 

At  this  stage,  you  direct  your  questions  towards  identifying  clues  from  the  information  you 
acquired  in  earlier  steps.  As  we  previously  defined,  clues  are  facts,  data,  or  information  that 


176  Schum,  D.A.  Evidence  and  Inference  for  the  Intelligence  Analyst.  Volume  1,  University  Press  of  America, 
Lanham,  MD,  1987,  p.  121. 

177  Elder,  L.  &  Paul,  R.  The  Miniature  Guide  to  the  Art  of  Asking  Essential  Questions,  4th  edn.  Foundation  for 
Critical  Thinking,  Dillon  Beach,  CA,  2006,  p.  3. 
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suggests  an  explanation  for  a  phenomenon.  This  explanation  becomes  a  tentative  hypothesis, 
which  can  then  be  applied  to  the  other  information  you  collected.  As  you  apply  the  hypothesis, 
you  identify  new  clues  that  lead  to  new  tentative  hypotheses.  When  applying  a  hypothesis  leads 
to  a  new  clue,  the  likelihood  increases  that  the  hypothesis  is  the  one  most  supported  by  the 
evidence  and  ultimately  leads  to  the  best  explanation. 

When  little  is  known,  how  do  we  know  that  the  hypotheses  we  advance  in  response  to 

clues  are  correct?  We  can  only  know  by  seeing  whether  they  lead  to  more  and  more  new 

clues.  If  they  do  they  are  likely  to  be  right  even  if  we  have  not  had  a  chance  to  test  them. 

Tests  are  not  the  only  way  by  which  to  find  out  how  good  a  hypothesis  is.178 

At  this  stage,  the  infonnation  associated  with  clues  becomes  evidence  as  it  linked  to  a  given 
hypothesis  and  increases  likelihood  of  that  hypothesis.  The  concept  of  marshalling  or  organizing 
evidence  is  frequently  a  topic  in  the  intelligence  literature. 

Different  ways  of  organizing  thought  and  evidence  may  lead  us  to:  (i)  ask  different 

questions  of  and  about  our  evidence,  (ii)  discover  different  evidence  and  hypotheses,  and 
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(iii)  draw  different  conclusions. 

We  might  not  even  have  noticed  certain  patterns  of  evidential  conflict  or  noticed  possible 

evidential  synergism  or  redundance  unless  we  had  organized  our  evidence  in  such  a  way 
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as  to  reveal  these  characteristics. 

The  emphasis  on  marshalling  evidence  in  the  intelligence  literature  stems  largely  from  those  who 
subscribe  to  the  “hypotheses  first”  analysis  process.  Although  FAC  is  not  based  on  the 
“hypotheses  first”  process,  some  of  the  concepts  presented  in  the  “hypotheses  first”  process  are 
useful  for  arraying  and  sorting  evidence.  The  FAC  approach  for  assigning  meaning  to  individual 
pieces  of  information  discussed  in  this  section  will  incorporate  some  of  the  methods  for 
marshalling  evidence  that  Schum  proposes.  We  will  discuss  using  these  approaches  more  in 
the  next  FAC  section,  “Assessment.” 

Hypotheses  that  are  not  based  on  clues  are  characterized  as  “wild  guesses.”  An  infonned  and 
experienced  person  may  generate  the  hypotheses,  but  they  are  still  guesses.  If  you’re  an  infonned 
and  experienced  person  you  may  be  basing  your  guess  on  prior  infonnation  acquisition  and 
knowledge  relevant  to  the  issue  at  hand.  This  may  result  in  a  better  guess  than  not  having 
information  and  knowledge  as  the  basis  for  your  guess.  However,  it  is  still  a  guess  until  you 
focus  any  needed  additional  infonnation  acquisition,  discovery  of  clues,  and  analysis  on  the 
particulars  of  the  specific  issue  you  are  addressing.  If  the  guess  is  not  conect,  you  cannot  obtain 
a  valid  answer  to  the  intelligence  question,  as  Heuer  observes: 

If  a  person  does  not  generate  the  correct  hypothesis  for  consideration,  obviously,  he  or 
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she  will  not  get  the  correct  answer. 


178  Lai  [unpaged]. 

17Q  Hughes,  F.  J.  &  Schum,  D.A.  The  Art  and  Science  of  the  Process  of  Intelligence  Analysis,  Joint  Military 
Intelligence  College,  Washington,  D.C.,  2003,  p.  37. 

180  Hughes  p.  38. 

181  Hughes  pp.  39-45. 

182  Lai  [unpaged] 

187  Heuer,  R.J.  Psychology’  of  Intelligence  Analysis,  Center  for  the  Study  of  Intelligence,  Central  Intelligence 
Agency,  Washington,  D.C.,  1999,  p.  96. 
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This  quote  underscores  the  necessity  of  searching  for  clues  that  lead  to  hypotheses.  A  cycle  of 
searching  for  clues  and  generating  hypotheses  continues  until  you  can  postulate  a  hypothesis  that 
considers  all  the  information  and  explains  most  of  the  information.  This  hypothesis  then  becomes 
your  belief  about  the  best  explanation  for  the  information.  The  hypothesis  becomes  the  tentative 
assessment  you’ll  use  to  address  the  intelligence  issue. 

In  some  cases,  a  single  hypothesis  may  not  explain  most  of  the  information.  You  may  have 
multiple  hypotheses  at  the  completion  of  your  analysis  that  are  supported  by  the  evidence  to 
varying  degrees,  resulting  in  multiple  explanations.  Consequently,  you  could  have  multiple 
beliefs,  not  all  believed  with  the  same  intensity,  resulting  from  your  tentative  hypotheses.  These 
beliefs  will  shape  your  assessment  and  how  you  eventually  communicate  that  assessment  to  your 
customers.  This  will  be  covered  in  more  detail  in  subsequent  sections  of  the  FAC. 

Recognizing  patterns  within  the  information  is  a  powerful  technique  for  finding  clues. 

Patterns  involve  the  composite  of  traits  or  features  that  are  characteristic  of  an  individual, 
a  group,  or  a  recognizably  consistent  series  of  related  acts.  Patterns  are  built  upon 
relationships  that  drive  human  behavior,  such  as  everyday  activities,  interactions,  and 
transactions  (in  a  very  low-level  sense),  and  strategy,  will,  and  decision-making  (on  a 
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higher  plane  of  thought  and  action). 

Based  on  everything  presented  in  the  FAC,  the  following  summarizes  the  FAC  approach  for 
assigning  meaning  to  the  individual  pieces  of  infonnation.  We’ve  indicated  the  relationship  to 
previous  FAC  discussions  in  parentheses. 

•  Whenever  you  interpret  any  information,  you  must  establish  an  unambiguous 
understanding  of  the  reason,  purpose,  and  goal  for  this  interpretation.  (Mindware) 

•  Based  on  that  understanding,  acquire  all  of  the  directly  and  indirectly  related 
infonnation. 

•  Read  and  understand  the  information  you  obtain.  (Mindware) 

•  Question  individual  segments  of  infonnation  to  detennine  the  originator’s  content 
(i.e.,  facts,  data,  or  information)  and  meaning.  (Thinking  dispositions) 

•  Identify  clues  from  the  aggregate  of  information  pertinent  to  the  purpose  or  goal. 
Base  tentative  hypotheses  on  the  clues  in  individual  pieces  of  infonnation. 
(Procedural  knowledge/cognitive  skills) 

•  Incorporate  the  hypotheses  best  supported  by  the  evidence  into  beliefs.  (Mindware) 

•  Because  you  may  have  changed  your  mindware  by  incorporating  new  beliefs  when 
you  analyze  new  information,  your  approach  to  considering  the  next  segment  of 
infonnation  may  be  different. 

•  Repeat  the  above  sequence  all  segments  of  information  you  retrieved. 

•  The  net  result  is  mindware  that  has  been  altered  both  in  content  and  beliefs. 

Now  we  can  get  on  with  the  actions  and  cognition  of  assigning  meaning:  acquiring  context, 
answering  the  what  question,  answering  the  why  question,  and  combining  these  three. 


184  Hall,  W.M.  &  Citrenbaum,  G.M.  Intelligence  Analysis:  How  to  Think  in  Complex  Environments,  Praeger  Security 
International,  Santa  Barbara,  CA,  2010,  p.  139. 
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Acquiring  Context:  Before  you  can  assign  meaning  to  information,  you  must  acquire  all  of  the 
contextual  information  that  relates  to  the  information  you  plan  to  evaluate.  The  activity  is 
illustrated  by  the  down  arrow  in  Figure  6,  labeled  “Info  Acquisition,  Context.”  To  acquire  all  the 
information,  do  the  following: 

•  During  the  acquisition  of  information,  you  were  asked  to  make  a  listing  of  people, 
places,  and  things  associated  with  the  search  topic  but  not  identified  in  the 
information  plan.  Now  this  listing  will  be  put  to  use. 

•  Devise  a  categorization  scheme,  which  might  correspond  to  the  search  topics  given 
in  the  Information  Acquisition  Plan  section. 

•  Assign  each  piece  of  relevant  information  to  a  category. 

•  Create  a  separate  listing  for  people,  for  places,  and  for  things.  Organize  the  listings 
based  on  the  category  of  the  relevant  infonnation  from  which  the  people,  places,  or 
things  were  obtained. 

•  For  each  listing,  accomplish  (at  least  conceptually),  an  information  plan  and  acquire 
the  information  as  described  in  the  Information  Acquisition  section.  We  recommend 
you  use  your  search  diary  for  this  additional  information  acquisition. 

•  Select  the  relevant  infonnation  from  that  acquired  in  step  5.  Your  relevance 
judgment  largely  consists  of  making  a  determination  about  which  retrieved 
infonnation  relates  to  the  specific  people,  places  and  things  for  which  you  are 
searching.  NOTE:  It  is  very  common  for  different  personalities  and  organizations  to 
have  the  same  name.  This  happens  less  for  equipment,  but  it  does  happen.  An 
equally  complicating  factor  is  that  the  same  personality,  organization,  or  equipment 
will  have  multiple  names  due  to  translations  and  transliteration  differences  or  the 
assignment  of  names  from  multiple  sources.  When  you  find  these  variances,  they 
become  the  basis  for  another  round  of  information  acquisition.  The  result  of  this 
step  is  a  collection  of  information  that  is  relevant  to  the  information  relevant  to  the 
intelligence  issue. 

•  Review  the  new  collection  of  information  to  detennine  if  any  people,  places  and 
things  emerged  that  you  didn’t  use  as  search  terms  previously.  If  so,  repeat  steps  4, 
5,  6,  and  7  for  the  new  search  tenns. 

•  Repeat  this  cycle  until  you  can  identify  no  new  people,  places  or  things  or  when 
you  have  retrieved  an  acceptable  level  of  information.  As  we  mentioned  before,  this 
is  a  judgment  you  may  need  to  make  based  on  time  limitations  or  because  the 
content  resulting  from  the  searches  seems  too  far  removed  from  the  intelligence 
issue  driving  your  analysis. 

The  information  acquired  by  the  above  approach,  coupled  with  the  relevant  infonnation  you 
acquired  previously,  constitutes  the  totality  of  the  information  based  on  what  you  now  know. 

You  use  this  totality  of  information  to  address  the  intelligence  issue.  This  is  not  to  say  that  as 
your  analysis  progresses  the  need  for  new  information  will  not  arise;  it  will.  But  for  now  you 
have  what  is  available  and  pertinent,  and  you  can  now  focus  on  assigning  of  meaning. 
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Answering  the  “What”  Question 

You  should  address  the  what  question  before  addressing  the  why  question  since  even  a  tentative 
answer  to  the  what  question  helps  answer  the  why  question.  Answering  the  what  question 
consists  of  two  activities: 

•  Reviewing  the  relevant  information  you  retrieved  during  information  acquisition  for 
substantive  content.  This  review  occurs  document  by  document. 

•  Establishing  the  timing  and  validity  of  the  whole  set  of  information. 

To  review  the  relevant  information  for  substantive  content,  do  the  following: 

1 .  Devise  a  categorization  scheme,  which  might  correspond  to  the  search  topics  given  in 
the  “Information  Plan”  section. 

2.  Assign  each  piece  of  relevant  information  to  a  category. 

3.  Select  a  category  for  the  what  question  you’re  answering.  You  can  begin  with  any 
category.  However,  by  this  time  you  are  likely  to  have  a  sense  that  a  particular  category 
may  be  more  relevant  to  the  intelligence  issue  than  others.  If  so,  you  can  select  that 
category  as  your  starting  point. 

4.  Sort  the  retrieved  documents  in  each  category  based  on  the  age  of  information 
associated  with  the  document.  If  the  document  itself  is  dated,  use  that  date  for  sorting. 

If  the  date  of  the  content  is  included  in  the  document,  use  that  date  for  sorting.  If  the 
content  isn’t  dated,  you’ll  conduct  a  refined  analysis  later  to  detennine  the  date  of  the 
content. 

5.  Use  the  oldest  document  as  a  starting  point.  Even  though  you  read  it  previously  to 
judge  its  relevance,  read  it  again  thoroughly.  Determine  the  content  of  the  document 
based  on  the  meaning  of  the  originator.  You  might  ask  yourself  questions  like  these  in 
order  to  determine  the  content  of  the  document: 

a.  What  is  the  substantive  nature  of  the  information  presented  in  the  document? 

b.  What  is  the  basic  message  the  document  communicates? 

c.  What  conclusions,  inferences,  ramifications,  or  observations  does  the  document 
contain? 

6.  Select  the  next  to  the  oldest  document.  Read  it  using  the  same  approach  you  used  to 
read  the  oldest  document. 

7.  Place  the  content  of  the  document  you  read  in  step  6  in  the  context  of  the  oldest 
document.  Some  typical  questions  you  might  ask  are: 

a.  How  does  the  content  of  this  document  differ  from  earlier  documents  that  dealt 
with  the  same  substantive  topic? 

b.  What  is  new  or  different  about  the  findings? 

c.  How  does  the  substantive  content  relate  to  the  issue  under  analysis? 

8.  Repeat  steps  5,  6,  and  7  for  every  document  in  each  category  and  for  all  categories. 
Remember  to  read  the  documents  in  chronological  order. 
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During  your  review  of  the  initial  set  of  relevant  documents,  you  may  identify  additional  search 
topics.  Use  the  approach  outlined  in  the  Information  Acquisition  section  to  obtain  additional 
documents  about  these  new  search  topics.  Once  you’ve  retrieved  new  documents,  repeat  the 
steps  above  to  assess  the  relevance  of  your  search  results. 

Once  you  have  identified  no  new  search  topics,  found  no  new  relevant  infonnation,  and  used  all 
of  the  preceding  steps  to  review  the  documents  on  hand,  you  have  completed  a  comprehensive 
review  of  the  documents  you  thought  were  relevant.  The  next  phase  of  your  review  is  to  conduct 
an  even  more  detailed  examination  of  the  relevant  documents  so  that  you  can  better  determine 
the  timing  of  events  and  validity  of  the  information  contained  in  the  relevant  documents.  Use  the 
following  approach  so  that  you  can  ascertain  consistency  for  both  the  information  in  a  single 
document  and  for  the  information  from  all  the  documents.  To  establish  the  timing  and  validity  of 
the  infonnation  you  retrieved,  do  the  following: 

1 .  You  organized  your  relevant  documents  previously  by  publication  or  infonnation  date. 
These  dates  become  even  more  important  when  you  are  taking  a  closer  look  at  the  timing 
of  an  event  or  situation.  It  is  possible  to  get  even  more  accurate  dating  by  looking  at 
submission  dates  or  investigating  the  typical  publication  delays  for  publications.  If  you 
can  identify  publication  or  submission  dates,  note  them. 

2.  Given  the  date  attributed  to  the  information  or  the  adjusted  dates,  is  a  logical  progression 
of  events  or  development  of  substance  evident? 

3.  If  you  examine  the  whole  of  the  relevant  information  and  the  dates  don’t  make  sense, 
what  adjustment  in  the  dates  are  required  for  a  logical  progression  to  exist? 

4.  Once  you  establish  a  sequence  of  events  for  the  relevant  information,  and  a  change  in  the 
development  of  substance  is  evident,  is  the  change  in  substantive  content  logical  or 
possible? 

5.  What,  if  any,  adjustment  in  substantive  content  is  required  to  establish  a  logical 
sequence? 

While  you  are  determining  the  validity  and  sequence  of  the  relevant  infonnation  you  retrieved, 
you  may  find  clues  that  ultimately  result  in  tentative  hypotheses.  These  hypotheses  may  lead  to 
other  clues  and  the  generation  of  new  tentative  hypotheses.  As  we  discussed  earlier,  hypotheses 
that  lead  to  more  clues  are  increasingly  likely  to  become  the  explanations  most  supported  by  the 
evidence.  From  a  cognitive  perspective,  beliefs  are  explanations  that  are  well-supported  by 
evidence. 

The  final  step  in  the  transition  of  a  tentative  explanation,  or  hypothesis,  to  a  belief  may  require 
you  to  acquire  more  information.  You  should  scrutinize  your  tentative  explanations  and  consider 
what  activities  would  result  from  the  explanations  if  they  were  to  be  true.  If  these  activities  were 
not  part  of  the  original  plan  for  information  acquisition,  it  is  critical  that  you  consider  them  new 
search  topics.  As  with  any  new  search  topics,  you  need  to  repeat  the  entire  process  of  acquisition, 
review  for  relevance,  and  assignment  of  meaning  for  these  newly  identified  search  topics. 

As  with  previous  FAC  sections,  we  stress  that  it  is  important  to  capture  your  thinking  while  you 
are  answering  the  what  question.  One  approach  for  capturing  your  thinking  might  be  to  record 
the  answers  to  all  the  questions  you  asked  about  the  infonnation  in  the  steps  above.  You  can  use 
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either  hard  copy  or  an  electronic  form  to  record  your  thinking.  However,  we  suggest  that  you 
capture  your  thoughts  in  an  electronic  form  because  using  that  format  will  be  helpful  to  you  later. 

Answering  the  Why  Question:  To  answer  the  why  question,  you  must  achieve  an  understanding 
of  the  objectives  and  motivations  associated  with  the  actions  of  the  people,  the  places,  and  the 
things  associated  with  the  intelligence  issue.  As  we  mentioned  previously,  if  you  know  the  habits 
of  people  acting  as  individuals  and  in  association  with  places  and  things,  you  can  establish  a  path 
to  gaining  understanding.  To  determine  what  these  habits  might  be,  you  need  to  derive  the 
following  from  the  information  you  acquired  on  people,  places,  and  things  associated  with  the 
intelligence  issue: 

•  People  -  history  of  activities,  associations  with  other  people  and  organizations, 
affiliations  with  places  and  things 

•  Places  -  history  of  activities,  organizational  structure,  including  subordination, 
mission,  and  people  affiliated  with  the  organization 

•  Things  -  place  of  development  or  acquisition,  person  or  persons  who  developed  or 
acquired  the  thing,  history  of  its  application  or  use,  processes  associated  with  its 
development,  acquisition,  or  use 

You  should  then  subject  this  information  on  habits  to  the  cycle  of  searching  for  clues  and 
generating  hypotheses  that  we  described  in  the  previous  section.  When  you  are  trying  to  answer 
the  why  question,  the  clues  you  find  and  the  hypotheses  that  result  are  related  to  the  motivation 
and  purpose  of  the  people,  places,  and  things  associated  with  the  intelligence  issue. 

Before  discussing  how  to  proceed  with  the  why  question,  we  have  to  explain  an  over 
simplification  in  Figure  6.  The  impression  created  by  Figure  6  is  that  you  start  with  the  people, 
place,  and  things  you  last  searched  and  then  progress  up  the  layers  as  you  do  your  analysis.  But, 
of  course,  analysis  is  not  a  linear  process;  it  is  highly  iterative.  While  we  are  recommending  a 
particular  sequence  for  your  analysis,  recognize  that  you  may  have  to  repeat  the  cycle  numerous 
times.  You  have  to  start  somewhere  and  analyze  the  people,  places,  and  things  you  identified 
during  information  acquisition  and  subsequent  searches.  You  repeat  the  steps  in  this  section  until 
you’ve  addressed  all  of  the  context  and  all  of  the  what  and  why  relationships  you  discover.  To 
illustrate,  just  imagine  the  following  relationships  among  people. 
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Figure  7:  Relationships 


In  this  example,  Person  1  was  identified  in  the  information  resulting  from  your  initial  search  for 
Information  Relevant  to  the  Intelligence  Issue.  Then  you  conducted  a  search  on  Person  1  and 
found  that  Person  1  has  associations  with  Persons  2,  3,  and  4.  So  to  establish  a  belief  on  the 
motivation  and  purpose  of  Person  1,  you  must  have  beliefs  on  the  motivation  and  purpose  of 
Persons  2,  3,  and  4.  You  next  focus  on  Person  2.  You  are  already  aware  that  Person  2  is 
associated  with  Person  1  (since  Person  1  led  you  to  Person  2).  You  also  are  aware  that  Person  2 
has  an  association  with  Persons  3  and  4  since  these  two  are  associated  with  Person  1 .  When  you 
then  search  specifically  on  Person  2,  you  find  another  piece  of  information  that  Person  2  is 
associated  with  Person  5  and  Person  4  in  a  different  relationship  than  the  relationship  with 
Person  1.  So  to  arrive  at  a  belief  for  the  motivation  of  Person  2,  you  must  have  beliefs  on  the 
motivation  and  purpose  of  Persons  1,3,4,  and  5,  including  the  motivations  and  purpose  involved 
with  Person  4’s  two  different  associations.  This  means  that  if  two  pieces  of  information  are 
associated  with  the  same  individual,  you  must  determine  the  why  for  each  of  the 
individual’s  different  relationships. 

As  the  limited  example  given  in  Figure  7  illustrates,  determining  the  why  for  an  intelligence  issue 
gets  complicated  because  any  individual’s  motivation  and  purpose  are  influenced  by  every  other 
person  with  whom  he  or  she  is  associated.  In  effect,  interdependency  exists  among  all  the  people 
in  this  example.  Detennining  the  motivation  of  any  one  person  depends  on  knowing  the 
motivation  and  purpose  of  all  the  others  with  whom  they  have  an  association.  The  situation 
becomes  even  more  complex  when  you  consider  the  interdependencies  among  people,  place  and 
things.  The  following  approach  provides  a  means  for  addressing  this  complexity: 

1 .  For  each  of  the  people,  places,  and  things  directly  associated  with  the  information  that  is 
relevant  to  the  intelligence  issue,  create  a  listing  of  the  people,  places  and  things.  Then 
create  a  listing  of  the  people,  places  and  things  directly  related  to  the  first  listing.  Then 
create  a  listing  of  the  people  places  and  things  related  to  second  listing.  Continue  until 
you  have  listings  covering  all  the  identified  people,  places,  and  things. 

2.  Select  a  person,  place,  or  thing  from  one  of  the  listings.  Answer  the  why  question  for  that 
person,  place  or  thing.  Since  you  will  not  have  any  beliefs  at  this  stage  for  the  associated 
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people,  place  or  things,  establish  a  tentative  hypothesis  for  motivation  and  purpose  based 
on  the  why  answers. 

3.  Select  a  second  person,  place  or  thing  and  answer  the  why  question.  In  this  instance,  the 
tentative  hypothesis  established  from  the  prior  step  can  complement  the  answers  to  the 
why  question  to  establish  a  tentative  hypothesis  for  motivation  and  purpose. 

4.  Repeat  this  process  for  all  the  people,  places  and  things  in  your  listings.  Remember  to  use 
all  prior  tentative  hypotheses  for  each  instance  when  establishing  each  new  tentative 
hypothesis. 

5.  When  you  have  established  tentative  hypotheses  for  all  of  the  people,  places  and  things  in 
the  listing  start  over  again  making  use  of  the  tentative  hypotheses.  Repeat  this  cycle  until 
the  hypotheses  for  the  motivation  and  purpose  are  consistent  and  complementary.  When 
you  generate  a  consistent  and  complementary  set  of  hypotheses,  these  hypotheses  are 
likely  to  be  the  most  supported  by  the  evidence  and  provide  the  best  explanation  of  the 
why.  As  we  mentioned  before,  from  a  cognitive  perspective,  that  explanation  then 
becomes  a  belief. 

6.  Capture  your  findings  for  each  person,  place  or  thing.  You  should  include  the  information 
discussed  earlier  that  you  used  to  establish  habits,  as  well  as  your  beliefs  on  motivation 
and  purpose.  How  you  chose  to  capture  this  information  is  up  to  you. 

Remember  that  the  final  step  in  the  transition  of  a  tentative  explanation,  or  hypothesis,  to  a  belief 
may  require  you  to  acquire  more  infonnation.  You  should  scrutinize  your  tentative  explanations 
and  consider  what  activities  would  result  from  the  explanations  if  they  were  to  be  true.  If  these 
activities  were  not  part  of  the  original  plan  for  infonnation  acquisition,  it  is  critical  that  you 
consider  them  new  search  topics.  As  such,  you  would  need  to  repeat  the  entire  process  of 
acquisition,  review  for  relevance,  and  assignment  of  meaning  for  these  newly  identified  search 
topics. 

9.5  Finally 

You  have  developed  a  belief  or  series  of  beliefs  for  what  the  activity  accomplished  (the  what)  for 
each  document  that  contains  information  relevant  to  the  intelligence  issue.  In  addition  you  have 
established  a  belief  or  series  of  beliefs  for  the  motivation  and  purpose  of  the  activity  (the  why). 
The  combination  of  these  beliefs  is  the  meaning  of  the  information  for  a  particular  document.  Of 
course  as  you  detennine  the  meaning  for  each  document,  new  judgments  about  meaning  might 
cause  you  to  change  prior  judgments.  Throughout  your  work  to  assign  meaning,  it  is  important 
that  you  are  vigilant  about  how  new  judgments  may  affect  prior  ones.  Be  prepared  to  make 
necessary  adjustments  in  the  meaning  you  assigned  to  infonnation  earlier  in  the  process  of 
determining  the  what  and  why  of  an  activity. 

As  we  have  discussed  repeatedly  in  the  FAC,  because  of  the  role  of  writing  plays  in  helping  to 
stimulate  your  cognition,  we  recommend  you  capture  the  meaning  of  each  relevant  document. 

We  suggest  you  use  a  method  that  works  well  for  you  but  stress  that  capturing  your  impressions 
in  writing  will  be  very  helpful. 

9.6  Summary 

This  section  has  provided  you  with  an  approach  to  discovering  the  unknown  by  assigning 
meaning  to  the  relevant  information  that  you  obtained  from  your  information  acquisition  efforts. 
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This  approach  is  based  on  three  key  concepts:  discovering  the  originator’s  meaning, 
understanding  physical,  cognitive,  and  emotional  habits  as  a  basis  for  prediction,  and 
understanding  the  structures  that  affect  the  actions  of  people  and  organizations. 

While  almost  all  intelligence  issues  relate  to  future  events,  knowing  and  understanding  both  past 
and  current  states  is  necessary  to  prepare  to  predict  the  future.  Clues  about  the  what  and  the  why 
are  the  means  to  discover  hidden  aspects  about  structure,  habits,  and  intent.  The  what  question 
can  be  answered  by  the  substance  of  the  relevant  information,  but  the  relevant  information  will 
rarely  provide  explicit  information  on  the  why  question.  However,  the  why  question  can  be 
answered  indirectly  by  analyzing  attributes  of  the  relevant  information  such  as  the  people, 
places,  and  things  associated  with  the  relevant  information. 

Determining  the  meaning  of  the  information  to  answer  the  what  and  the  why  for  each  piece  of 
information  depends  on  understanding  the  associated  context.  The  challenge  for  the  analyst  is  to 
create  the  context  of  the  originator  of  the  information.  To  understand  context,  you  must  acquire 
additional  information  based  upon  the  new  people,  places,  and  things  identified  during  the 
information  acquisition.  Ideally  this  search  continues  until  no  new  people,  places,  or  things  are 
identified.  Then  you  can  focus  on  the  assignment  of  meaning  to  answer  the  what  and  the  why 
questions.  However,  analysis  is  highly  iterative  and  involves  the  repetition  of  all  the  cycles  of  the 
FAC  approach  for  assigning  meaning. 

By  comprehensively  reading  the  acquired  infonnation  and  asking  questions  about  it,  you 
generate  additional  clues.  These  clues  ultimately  result  in  tentative  hypotheses,  and  the 
information  supporting  those  hypotheses  becomes  evidence  as  it  is  linked  to  a  given  hypothesis. 
The  tentative  hypotheses  that  are  most  supported  by  the  evidence  become  beliefs  about  the  what 
and  the  why.  The  combination  of  these  beliefs  is  the  meaning  of  information  for  a  given 
document. 

In  the  next  section,  “Assessment,”  you  will  use  the  beliefs  that  you  established  for  the  meaning 
of  each  piece  of  information.  Y ou  use  these  beliefs  to  form  your  understanding  of  the  current 
status  associated  with  your  intelligence  issue  and  then  to  develop  a  prediction  that  addresses  the 
intelligence  issue. 
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10.0  ASSESSMENT 


I  take  what  I  see  work.  I'm  a  strict  believer  in  the  scientific  principle  of  believing  nothing,  only 
taking  the  best  evidence  available  at  the  present  time,  interpreting  it  as  best  you  can,  and  leaving 
your  mind  open  to  the  fact  that  new  evidence  will  appear  tomorrow. 

-Adam  Osborne 

Study  the  past,  if  you  would  divine  the  future. 

-Confucius 

10.1  Introduction 

At  this  stage  you  have  finished  the  work  of  assigning  meaning  to  all  the  information,  piece  by 
piece,  that  you  judged  as  being  relevant  to  the  intelligence  issue  under  analysis.  The  meaning 
you  assigned  was  based  on  answering  the  what  and  the  why  for  each  piece  of  information  given 
the  associated  context.  It  involved  a  rigorous  process  of  finding  clues  that  led  to  tentative 
hypotheses,  testing  the  hypotheses  using  available  information,  using  supported  hypotheses  to 
find  more  clues  that  then  lead  to  even  more  new  hypotheses  for  testing.  You  repeated  this  cycle 
of  finding  clues  based  on  acquired  infonnation  leading  to  supported  hypotheses,  leading  to  more 
clues,  until  no  new  clues  or  hypotheses  resulted.  The  tentative  hypotheses  that  were  most 
supported  by  the  evidence  became  beliefs  about  the  what  and  the  why.  These  beliefs  reflect  the 
substance  and  motivation  associated  with  the  content  reflected  in  each  individual  piece  of  the 
relevant  infonnation.  The  beliefs  that  you  developed  during  the  assignment  of  meaning  now 
become  an  important  input  to  your  assessment  of  the  intelligence  issue.  This  section  discusses 
how  to  carry  out  the  assessment  of  the  intelligence  issue. 

10.2  First,  Some  Terminology 

The  intelligence  literature  is  replete  with  the  polysemy,  or  many  definitions,  of  such  tenns  as 
evidence,  hypothesis,  assumptions,  beliefs,  and  conclusions.  The  net  result  is  that  agreements 
and  disagreements  which  seem  to  exist  regarding  the  cognition  of  analysis  do  not,  in  fact,  exist. 
This  is  the  consequence  of  using  different  definitions  for  crucial  words  that  describe  analysis. 

For  use  in  the  FAC,  definitions  for  such  words  were  provided  in  the  Assignment  of  Meaning 
section.  Since  these  words  are  heavily  used  in  this  section  we  recommend  that  you  re-familiarize 
yourself  with  the  FAC  definitions. 

We  do  introduce  some  new  terms  in  this  section.  The  beliefs  that  you  developed  while  you  were 
assigning  meaning  are  called  beginning  beliefs  in  this  section  of  the  FAC.  When  you  assigned 
meaning,  information  was  the  input  to  your  cognition.  During  assessment,  the  inputs  to  your 
cognition  are  the  beginning  beliefs  you  developed  while  assigning  meaning.  The  objective  of 
doing  assessment  is  to  integrate  and  correlate  these  beliefs  so  that  you  generate  an  overall 
belief.  This  overall  belief  forms  the  basis  for  your  judgments  and  conclusions;  these  judgments 
and  conclusions  are  your  assessment  of  the  intelligence  issue. 
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10.3  Marshalling  Evidence 

In  the  “Assignment  of  Meaning”  section,  we  discussed  the  concept  of  marshalling  evidence 
during  the  assignment  of  meaning.  Schum  best  describes  the  importance  of  marshalling: 

But  the  marshaling  of  thoughts  and  evidence  in  inference  tasks  has  far  greater  importance 
than  mere  convenience.  In  addition,  there  are  issues  that  arise  in  the  study  of  such  tasks 
that  are  anything  but  trivial.  Thoughts  and  evidence  organized  or  juxtaposed  in  one  way 
can  lead  to  significant  insights  that  do  not  flash  before  us  when  these  same  thoughts  and 

185 

evidence  are  organized  in  other  ways. 

Schum  applies  the  common  usage  of  “marshalling  of  evidence”  to  sorting  and  organizing 
evidence.  The  value  of  sorting  and  arranging,  however,  applies  to  more  than  examining 
evidence.  For  example,  in  the  previous  section  we  suggested  using  such  marshalling  to  identify 
clues.  In  this  section,  we  recommend  the  use  of  sorting  and  making  arrays  in  order  to  organize 
your  beliefs. 

Here  are  some  of  the  ways  of  sorting  and  creating  arrays  proposed  in  the  literature: 

•  Marshalling  by  sources 

•  Creating  event  chronologies 

•  Marshalling  by  specific  questions 

•  Marshalling  by  hypotheses 

•  Marshalling  by  arguments 

•  Marshalling  for  elimination 

1 

•  Constructing  scenarios  or  stories  from  evidence 

Schum  suggests  categorization  as  the  primary  criterion  for  marshalling.  Some  examples  of 
categorization  are: 

•  People 

•  Places 

•  Substance/what 

•  Motivation/why 

Within  each  of  these  categories  Schum  suggests  event  chronologies  as  the  most  common  basis 
for  sorting  and  arraying.  Changes  in  the  categories  as  a  function  of  time  are  a  powerful  means  of 
achieving  insight.  Arraying  of  the  aggregation  of  all  the  activities  as  function  of  time  can  be 
equally  useful. 

You  can  apply  all  of  these  arraying  techniques  to  beginning  beliefs.  Once  you  have 
accomplished  this,  you  need  to  examine  the  various  arrays  of  beliefs  for  clues  as  we  described  in 
the  prior  FAC  section.  The  objective  of  this  examination  is  to  find  beliefs  that  describe  a 
common  intent  of  what  and  why  for  a  cluster  of  endeavors.  When  you  assign  a  common  intent  to 
a  cluster  of  beliefs  you  are,  in  effect,  developing  a  tentative  hypothesis  which  provides  an 


185  Schum,  D.A.  "Marshaling  Thoughts  and  Evidence  during  Fact  Investigation,"  South  Texas  Law  Review,  Vol.  40, 
No.  2,  1999,  p.401. 

186  Hughes  pp.  39-45. 
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explanation  for  these  endeavors.  You  can  then  use  this  hypothesis  to  find  more  clues.  If  you  can 
find  enough  clues,  you  can  consider  the  hypotheses  to  be  well  supported  by  the  underlying 
beginning  beliefs.  The  hypothesis  then  becomes  a  new  belief,  an  intermediate  belief  that 
incorporates  several  of  the  beginning  beliefs.  You’ll  continue  this  process  until  you  incorporate 
all  of  the  beginning  beliefs  into  intennediate  beliefs.  Finally,  you  need  to  incorporate  all  of  the 
intennediate  beliefs  into  an  overall  belief.  This  overall  belief,  along  with  the  beliefs  on  which  it 
is  based,  provides  an  explanation  for  the  historical  activities  of  the  target  as  well  as  the  target’s 
current  status. 

The  possibility  exists  that  you  cannot  find  clues  that  lead  to  a  hypothesis.  In  effect,  you  cannot 
put  forth  an  explanation  for  the  activities  under  analysis.  When  you  encounter  this  situation,  you 
have  two  courses  of  action.  The  first  is  to  “guess”  at  a  hypothesis  using  the  techniques  described 
in  the  prior  FAC  section.  Various  activities,  some  of  which  may  not  have  driven  your  original 
information  acquisition,  will  be  associated  with  this  “guessed”  hypothesis.  If  so,  use  should  use 
these  activities  as  the  search  topics  for  a  new  effort  to  acquire  infonnation.  The  information  you 
retrieve  may  provide  additional  insight  or  evidence  that  supports  your  “guessed”  hypothesis. 

You  should  then  review  the  information  you  acquire  using  these  new  search  topics  and  identify 
any  relevant  infonnation.  During  this  review,  the  basis  for  your  relevance  judgment  is  the 
relationship  between  the  infonnation  content  and  the  “guessed”  hypothesis.  Subject  all  of  the 
relevant  information  to  the  same  approaches  you  used  when  you  reviewed  the  information 
relevant  to  the  intelligence  issue.  If  the  results  of  these  approaches  support  the  “guessed” 
hypothesis,  you  can  adopt  the  hypothesis  as  an  intermediate  belief  and  continue  with  generating 
more  intermediate  beliefs  and  merging  intermediate  beliefs  into  a  final  belief. 

If  the  approach  described  above  does  not  result  in  a  usable  intermediate  belief,  the  strong 
implication  is  that  an  explanation  for  what  and  why  that  encompasses  all  the  endeavors  related  to 
the  intelligence  issue  does  not  exist.  Nevertheless,  you  can  address  the  intelligence  issue  based 
on  this  lack  of  an  overall  belief.  This  is  the  second  course  of  action  when  you  cannot  find  clues 
that  lead  to  a  hypothesis. 

If  the  relevant  information  covers  an  extended  period  of  time,  for  example  more  than  5  years, 
you  would  expect  to  see  a  change  in  purpose  and  motivation  during  that  period.  Thus,  when  you 
examine  the  beginning  beliefs  as  a  function  of  time,  you  need  to  be  sensitive  to  such  changes. 
When  you  note  changes  in  purpose  and  motivation,  you  need  to  separately  apply  the  process  of 
generating  beliefs  to  the  time  period  before  the  change  occurred.  You  then  generate  an  overall 
belief,  a  historical  belief,  for  the  time  period  prior  to  the  shift  as  well  as  a  current  overall  belief. 

Historical  overall  beliefs  provide  a  perspective  on  events  that  occur  subsequent  to  the  time  period 
covered  by  the  belief  and  help  in  the  interpretation  of  these  events.  However,  the  value  of  the 
historical  overall  beliefs  is  the  opportunity  to  calibrate  your  process  of  generating  beliefs  with 
reality.  You  can  compare  the  known  actions  of  a  country  or  group  with  purposes  and  intents 
derived  from  the  analysis  process.  Obviously,  the  higher  the  degree  of  correlation  between  these 
two  factors,  the  more  confidence  you  have  in  the  beliefs  related  to  the  current  purpose  and  intent. 

As  a  final  step,  scrutinize  the  overall  belief  to  identify  any  activities  that  result  from  the  belief 
that  were  not  part  of  the  original  plan  for  information  acquisition.  If  you  identify  any  activities, 
they  constitute  new  search  topics.  As  such,  you  must  repeat  the  entire  process  of  acquisition, 
assignment  of  meaning  and  marshalling  of  beliefs  for  these  newly  identified  topics. 
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Once  you  complete  the  last  step  and  make  any  adjustments  to  the  overall  belief,  you  have  an 
explanation  for  the  current  status  of  a  country  or  group  activities  in  areas  related  to  the 
intelligence  issue.  This  overall  belief  and  all  of  its  supporting  beliefs  and  evidence  are  the  basis 
you’ll  use  for  developing  an  assessment  for  the  intelligence  issue. 

As  with  other  FAC  sections,  we  strongly  suggest  that  you  use  writing  to  capture  the  process  and 
results  of  fonning  beliefs.  Your  written  description  should  include  a  description  of  the  evolution 
from  beginning  beliefs  to  overall  beliefs  and  the  basis  for  each  of  the  beliefs.  If  you  generated 
historical  beliefs,  you  should  also  include  them  as  well  as  all  the  correlations  identified  between 
these  beliefs  and  subsequent  events. 

10.4  The  Assessment  for  the  Intelligence  Issue 

When  you  acquire  your  final  belief  regarding  all  of  the  activities  related  to  the  intelligence  issue, 
you  are  at  the  end  point  of  making  sense  of  the  available  infonnation.  The  last  step  is  the 
application  of  your  beliefs  and  the  underlying  information  to  develop  an  assessment  for  the 
intelligence  issue.  However,  before  doing  so,  you  should  acknowledge  that,  as  a  result  of 
following  the  FAC,  you  are  a  changed  person.  You  have  learned  new  information  and  acquired 
new  beliefs,  all  of  which  have  substantially  altered  your  mindware.  This  change  in  mindware 
does  affect  the  interpretation  and  value  you  assign  to  information.  Also,  the  approaches 
presented  in  FAC  have  a  basic  sequential  characteristic.  On  numerous  occasions  the  FAC 
recommends  that  you  identify  new  topics  for  investigation  and  subject  these  new  topics  to  the 
complete  FAC  approach.  However,  you  have  not  been  asked  to  review  what  was  accomplished 
in  prior  sections  before  addressing  the  intelligence  issue.  The  time  has  come  for  you  to  do  this.187 

The  change  in  your  mindware  can  significantly  affect  the  work  you  did  in  three  sections  of  the 
FAC:  developing  the  intelligence  issue  statement,  creating  the  analysis  plan,  and  performing 
information  acquisition.  As  with  prior  FAC  sections,  it  is  important  to  reexamine  prior  efforts  as 
your  mindware  changes  throughout  the  analysis  process.  The  following  discussion  describes  how 
these  changes  in  mindware  might  require  additional  work  in  each  of  these  areas. 

10.4.1.  Intelligence  Issue  Statement 

The  starting  point  of  all  intelligence  analysis  is  the  expression  of  a  need  for  such  analysis,  in 
effect  a  statement  of  an  intelligence  issue.  Since  the  intelligence  issue  has  a  domino  effect  on  all 
that  follows,  you  need  an  unambiguous  understanding  of  the  intelligence  issue.  This  includes 
why  the  need  exists  and  how  the  requestor  plans  to  use  the  intelligence  assessment  once  you 
provide  it.  Your  interaction  with  the  source  of  the  issue  is  very  beneficial;  however  your  efforts 
to  gain  a  full  understanding  may  be  impeded  by  an  imperfect  knowledge  of  the  nature  or 
characteristics  of  the  issue.  This  may  also  be  the  case  if  you  initiate  the  intelligence  issue. 

Clearly,  at  the  end  of  the  analysis  effort,  you  have  a  greater  level  of  knowledge  and 
understanding  than  you  did  in  the  initial  stages  of  your  analysis.  Did  your  understanding  of  the 
intelligence  issue  change?  If  so,  you  need  to  consider  whether  this  change  in  understanding 
results  in  a  need  for  information  beyond  that  which  you  have  already  acquired  in  the  completed 


187This  description  can  convey  the  image  of  mindware  changing  in  discrete  steps  at  various  intervals  during  the  FAC 
procedures.  Obviously,  this  is  an  incorrect  image  since  mindware  changes  continuously  and  your  analysis  is 
potentially  affected  by  every  change.  From  a  practical  standpoint  one  cannot  redo  all  the  cognition  done  prior  to  a 
change  in  mindware.  What  we  do  propose  in  the  FAC  is  to  hold  in  abeyance  the  reexamination  of  those  portions  of 
FAC  that  are  very  sensitive  to  mindware  until  you  achieve  what  you  deem  is  the  final  mindware  regarding  the 
intelligence  issue. 
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effort.  If  you  identify  the  need  to  acquire  more  information,  you  must  iterate  through  the  FAC  for 
this  newly  identified  information  before  you  develop  your  assessment  for  the  intelligence  issue. 

10.4.2.  Analysis  Plan 

Most  intelligence  issues  relate  to  actions  on  the  part  of  others.  To  accomplish  these  actions  the 
entities  involved  must  have  the  capability  to  accomplish  the  action,  the  motivation  to  take  action, 
and  finally,  to  actually  make  a  decision  to  take  the  action.  It  follows  then  that  the  analysis  must 
determine  the  level  of  capability  possessed  by  an  entity  as  well  as  efforts  to  change  its 
capabilities,  the  motivations  of  an  entity,  and  how  decisions  are  made,  including  inputs  to  the 
decision  making  an  entity.  The  analysis  plan  identifies  the  effort  and  approach  required  to  make 
these  determinations.  Based  on  the  organizational  allocation  of  roles  and  responsibilities,  the 
analysis  plan  also  identifies  who  will  make  such  detenninations.  To  a  large  degree  the  analysis 
plan  is  based  on  the  pertinent  knowledge  possessed  at  the  initiation  of  the  analysis  effort.  As  we 
mentioned  before,  this  knowledge  is  likely  to  be  less  comprehensive  that  what  you  will  know  at 
the  end  of  the  analysis  effort.  Y our  increased  knowledge  may  reveal  omissions  in  your  original 
plan  and  necessitate  the  acquisition  of  additional  information.  As  we  described  above  with  issue 
statement,  the  acquisition  of  additional  infonnation  means  another  iteration  through  the  FAC 
before  you  can  develop  an  assessment  for  the  intelligence  issue. 

10.4.3.  Information  Acquisition 

The  change  in  your  mindware  affects  an  important  facet  of  information  acquisition:  the  selection 
of  information  that  you  deemed  to  be  relevant  to  the  intelligence  issue.  You  may  have  rejected 
some  information  as  being  non-relevant  that  you  would  now  judge  as  being  relevant  because  of 
the  change  in  your  mindware.  As  a  practical  matter  you  cannot  reexamine  all  of  the  rejected 
information  because  of  volume  and  the  likelihood  you  didn’t  retain  the  rejected  information.  As 
an  alternative  you  can  do  an  introspective  examination  to  identify  the  changes  in  mindware  that 
could  affect  your  relevance  judgments.  If  you  identify  such  changes  you  can  use  them  to  search 
for  information.  You  can  review  any  infonnation  you  acquire  to  identify  relevant  information. 

As  in  the  issue  statement  and  the  analysis  plan,  you  then  subject  all  the  newly  identified  relevant 
information  to  the  FAC. 

Once  you  have  re-examined  your  mindware  and  completed  all  of  the  related  actions  we  describe 
above,  the  possibility  exists  that  your  mindware  has  changed  yet  again.  Does  this  mean  that  you 
should  repeat  what  is  described  above?  The  answer  is:  it  just  depends.  Only  you  can  judge  the 
magnitude  of  change  that  has  occurred  and  what,  if  any,  new  insights  have  resulted.  The 
possibility  does  exist  that  you  will  need  to  repeat  the  above  actions  again. 

Once  you  achieve  a  final  overall  belief,  you  are  ready  to  develop  your  assessment  for  the 
intelligence  issue. 

10.5  Prediction 

Despite  how  well  you  know  the  current  status  of  the  activities  and  the  intent  of  the  target  entities, 
or  how  confident  you  are  of  this  knowledge,  the  probability  is  high  that  you  do  not  yet  have  an 
assessment  for  the  intelligence  issue.  Almost  all  intelligence  issues  relate  to  a  future  state  of 
activities,  developments,  and  intent.  Thus,  you  must  now  use  your  knowledge  of  the  current 
status  to  predict  what  will  result  from  these  endeavors. 

Generally,  predictions  about  the  behavior  of  a  target  fall  into  one  of  three  categories: 
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•  A  decision  has  been  made  regarding  a  future  objective.  Efforts  are  underway 
towards  achieving  this  objective. 

•  A  decision  about  a  future  object  has  not  yet  been  made  but  efforts  to  build  the 
capability  to  achieve  the  objective  once  a  decision  has  been  made  are  underway. 

•  No  decision  has  been  made  and  no  efforts  are  underway  toward  building  the 
necessary  capabilities  for  some,  as  yet,  undefined  objective. 

Predictions  that  fall  into  the  first  category  are  the  easiest  to  make.  From  your  understanding  of 
the  current  status  of  the  target  you  likely  know  the  length  of  time  the  efforts  have  been  underway 
as  well  as  what  the  target  has  accomplished  to  date.  Using  the  historical  record  of  similar 
endeavors  as  an  analog,  plus  any  special  consideration  associated  with  the  current  endeavor,  you 
can  make  a  prediction  for  the  time  required  for  completion  of  the  effort. 

When  the  target  has  not  made  a  decision,  but  supporting  efforts  are  underway,  the  primary  basis 
for  the  prediction  is  the  use  of  analogs.  You  can  use  historical  information,  modified  based  on 
the  relative  complexity,  the  status  of  supportive  activities,  and  any  special  considerations 
associated  with  the  current  endeavor  that  are  not  associated  with  the  analog  to  make  your 
prediction.  You  can  qualify  your  prediction  of  the  date  when  the  objective  will  be  achieved  in 
terms  of  time  needed  after  the  target  makes  the  decision  to  move  forward. 

Predictions  in  the  last  category  are  the  most  difficult  to  make.  In  effect,  you  have  found  no 
information  that  acts  as  an  indicator  of  efforts  that  lead  to  the  objective.  This  condition  will 
often  be  reflected  in  the  buildup  of  intermediate  and  overall  beliefs.  The  inability  to  arrive  at  an 
overall  belief  is  highly  indicative  of  a  lack  of  planned  efforts  directed  towards  a  specific 
objective.  While  in  such  cases  it  may  be  prudent  not  to  make  a  prediction,  the  nature  of 
intelligence  analysis  typically  requires  one  to  be  made.  Consequently,  making  a  prediction  may 
be  unavoidable.  In  such  circumstances  the  only  recourse  is  to  use  the  experience  of  other 
countries  or  groups  that  have  accomplished  the  objective.  You  have  to  modify  the  experiences 
of  other  countries  or  groups  based  on  conditions  as  they  exist  in  the  target  entity. 

A  very  valuable  technique  for  aiding  prediction  is  the  identification  of  the  milestones  of  actions 
or  accomplishment  leading  up  to  the  accomplishment  of  the  objectives.  The  required  milestones 
aid  in  the  establishment  of  timelines,  serve  as  indicators  of  future  progress,  and  reveal 
benchmarks  for  comparing  anticipated  accomplishments  versus  actual  accomplishment. 
Knowledge  of  benchmarks  permits  you  to  adjust  your  predictions  at  the  earliest  possible  date  if 
you  determine  that  the  benchmarks  have  changed. 
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10.6  Uncertainty 

Universal  agreement  exists  that  the  output  from  intelligence  analysis  has  a  degree  of  uncertainty. 
The  output  of  intelligence  analysis  is  based  on  incomplete  information  and  the  target  of  the 
intelligence  analysis  activity  is  uncooperative,  to  say  the  least.  As  a  result,  a  frequent  topic  in  the 
intelligence  literature  is  how  to  deal  with  uncertainties.  Tools  and  techniques  for  quantification 
of  uncertainty  are  an  especially  popular  topic. 


QUANTIFYING  UNCERTAINTY 

Under  the  premise  that  no  conclusion  can  be  better  than  the  evidence  upon  which  it  is 
based,  Schum  has  written  extensively  and  comprehensively  on  the  role  of  evidence  in 
supporting  hypotheses  and  conclusions  (Schum  1987).  He  deals  with  such  topics  as 
assessing  the  value  of  evidence,  source  credibility,  conflicting  evidence,  redundant 
evidence,  and  missing  evidence.  In  addition  he  provides  information  on  three  statistical 
techniques  for  dealing  with  uncertainties;  Bayesian,  Baconian  and  Shafer-Dempster 
methods  of  evidence  aggregations.  At  the  same  time  Schum  recognizes  the  practical 
difficulties  of  using  this  information  in  judging  the  degree  of  uncertainty: 

Conventional  courses  in  logic,  probability,  and  statistics  do  not  prepare  a  person  for  the 
task  of  drawing  conclusions  based  on  masses  of  evidence  whose  items  suggest  many, 
often  complex  and  interrelated,  lines  of  arguments  on  hypotheses  of  interest.  (Hughes  & 
Schum  2003:  2) 

Trying  to  keep  track  of  all  uncertainties  associated  with  sources  of  evidence,  much  less  to 
assess  and  combine  them  all,  would  be  inferentially  paralyzing.  Only  you  can  decide  what 
you  will  believe;  all  our  present  studies  can  do  is  to  alert  you  to  questions  you  can  ask  in 
the  process  of  deciding  what  you  will  believe.  (Schum  1987:  125) 

While  we  do  not  question  the  importance  of  evidence,  to  consider  evidence  as  the  sole  and 
dominant  factor  in  determining  uncertainty  is  ignoring  the  other  inputs  to  the 
inference/analysis  process:  hypotheses  and  beliefs.  Since  which  information  is  considered 
evidence  is  dependent  on  the  hypothesis  under  consideration,  the  selection  of  an 
erroneous  hypothesis  can  result  in  a  very  uncertain  outcome.  Also,  the  beliefs  used  in  the 
inference  process  are  based  on  other  inference  processes.  You  cannot  judge  the 
uncertainty  of  the  result  of  a  particular  inference  process  without  a  consideration  of  the 
uncertainty  of  preceding  and  related  inference  processes.  Thus,  any  expression  of 
uncertainty  regarding  the  results  of  an  inference  process  is  based  on  series  of  subjective 
judgments. 
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While  the  desire  to  quantify  the  degree  of  uncertainty  is  understandable,  what  you  are  really 
being  asked  to  do  is  to  express  the  strength  of  your  belief  in  the  likelihood  of  an  outcome.  You 
are  being  asked  to  judge  findings,  conclusions  and  predictions  that  are  based  on  a  multitude  of 
beliefs.  This  can  be  best  illustrated  by  a  brief  review  of  the  approaches  used  in  getting  from 
information  to  meaning. 

The  basic  FAC  approach  for  detennining  the  meaning  of  infonnation  is: 

•  Acquire  all  the  infonnation  relevant  to  your  purpose  (the  intelligence  issue). 

•  Array  the  information  so  that  you  can  identify  clues  that  suggest  an  explanation  for 
the  information  acquired. 

•  Your  purpose  now  changes  to  testing  the  hypotheses.  By  applying  your  mindware, 
identify  infonnation  that  supports  the  validity  of  a  tentative  explanation 
(hypothesis).  Such  infonnation  becomes  evidence  for  the  hypothesis. 

•  If  the  tentative  hypothesis  seems  to  be  true,  consider  what  other  infonnation  could 
be  evidence.  Seek  this  information  and,  if  you  find  it,  subject  it  to  the  FAC 
approaches. 

•  Based  on  the  amount  and  nature  of  the  evidence  you  have  at  hand,  determine  the 
likelihood  that  the  hypothesis  is  true  and  valid. 

•  If  the  likelihood  is  high,  convert  the  hypothesis  to  a  belief  and  incorporate  it  into 
your  mindware. 

•  Apply  your  revised  mindware  to  subsequent  approaches  for  determining  meaning. 

Apply  this  approach  to  all  information  relevant  to  the  intelligence  issue  and  to  all  infonnation 
that  is  relevant  to  the  relevant  information. 

We  have  stated  that  an  explanation  for  all  the  relevant  infonnation  to  the  intelligence  issue 
depends  on  assigning  meaning  for  each  piece  of  information.  The  meaning  assigned  is  based  on  a 
belief  as  to  the  “what”  and  “why”  for  each  piece  of  information.  One  cannot  predict  how  many 
pieces  of  relevant  information  will  be  found  for  an  intelligence  issue,  but  for  illustration  purposes 
let’s  say  100  pieces  of  information  are  relevant.  Thus  the  assignment  of  meaning  will  be  based 
on  100  conclusions  of  the  “what”,  100  conclusions  of  the  “why”,  and  100  conclusions  on  the 
meaning  of  the  combination  of  the  “what”  and  the  “why.”  However,  as  we  stated  in  a  prior 
section,  detennining  the  “why”  depends  on  determining  the  “what”  and  “why”  of  information 
that  is  relevant  to  the  relevant  infonnation.  Again,  the  amount  of  relevant  to  relevant  information 
is  unpredictable  but  will  be  much  greater  than  the  amount  of  relevant  infonnation.  In  our 
example,  one  could  easily  identify  500  pieces  of  relevant  to  relevant  information.  The  end  result 
is  that  the  assignment  of  meaning  to  information  germane  to  the  intelligence  issue  could  be  based 
on  conclusions  derived  from  hundreds,  if  not  thousands,  of  interdependent  beliefs. 

The  statistical  approach  for  calculating  uncertainty  is  to  multiply  the  uncertainty  of  one  factor 
times  the  uncertainty  of  all  the  other  associated  factors.  The  application  of  this  approach  to  all 
the  beliefs  described  above  produces  a  meaningless  and  unusable  result.  Even  if  you  were  very 
certain  about  each  and  every  belief  you  used  in  assigning  meaning,  the  multiplication  of  that 
value  thousands  of  times  results  in  an  extremely  small  value  of  certainty.  That  value  is  highly 
unlikely  to  agree  with  the  degree  of  certainty  that  you  would  subjectively  assign. 
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So  what  can  you  do?  How  can  you  reconcile  quantifying  the  degree  of  uncertainty  with  your 
subjective  sense  of  uncertainty?  Perhaps  the  explanation  lies  in  the  following: 

Phillips  argues  that,  though  probabilities  are  numbers,  uncertainty  is  a  feeling:  in  short, 

there  are  no  "true"  probabilities  residing  in  our  heads,  only  diffuse  feelings  of 
•  188 
uncertainty. 

The  situation  is  exacerbated  by  the  contrast  between  the  FAC  analysis  approaches  and  those 
typically  described  in  the  intelligence  literature.  The  latter  focuses  on  evidence  that  relates  to  the 
final  conclusion;  the  FAC  arrives  at  conclusions  from  a  scaffolding  of  beliefs  where  only  the 
lower  level  beliefs  are  likely  to  be  directly  dependent  on  evidence. 

Having  conclusions  based  on  hundreds  or  more  interdependent  beliefs  is  a  positive,  not  negative, 
factor  in  judging  uncertainty.  As  we  mentioned  in  a  prior  section  of  the  FAC,  one  of  the  means 
of  revealing  secrets  is  to  understand  the  structure  of  whatever  is  being  kept  secret.  The  structure 
provides  order,  consistency,  and  relationships.  An  evaluation  of  the  coherence  and  consistency  of 
the  beliefs  that  result  from  your  analysis  in  establishing  and  defining  a  structure  is  a  very 
effective  means  of  deriving  insight  into  the  certainty  of  your  conclusions. 

Comparing  your  analysis  results  with  reality,  which  we  mentioned  earlier,  is  another  method  for 
understanding  your  degree  of  certainty.  If  you  are  able  to  make  accurate  predictions  of  past 
events  based  on  analysis  of  information  that  preceded  the  event,  your  ability  to  understand  what 
is  happening  and  what  will  happen  is  very  much  enhanced. 

As  we  discussed  earlier  addressing  the  intelligence  issue  involves  fonnulating  an  explanation  for 
the  current  status  of  a  country  or  group  activities  in  areas  related  to  the  intelligence  issue  and 
making  predictions  based  on  this  explanation.  You  have  to  express  your  degree  of  uncertainty 
about  both. 

The  common  practice  in  the  intelligence  community  is  to  express  uncertainty  in  tenns  of  verbal 
expressions.  For  more  than  40  years,  efforts  have  been  underway  to  quantify  uncertainty 
because  it  is  known  that  different  people  assign  different  degrees  of  uncertainty  to  the  same 
verbal  expression.  Despite  all  these  efforts  the  practice  of  verbal  expressions  for  uncertainty  is 
still  dominant,  perhaps  for  the  reasons  given  by  Phillips.  Regardless  of  the  technique  used  and 
means  of  expression,  a  statement  of  uncertainty  is  still  a  subjective  judgment  by  the  analyst  who 
performed  the  analysis.  Only  you  know  how  much  infonnation  was  supportive  of  the  analysis, 
how  well  supported  hypotheses  were  based  on  evidence,  the  internal  consistency  of  the 
hypotheses  you  generated  and  how  strongly  you  believe  the  beliefs  that  resulted  from  this 
analysis.  Ultimately  when  you  prepare  the  product  for  the  customer,  you  will  use  whatever 
system  has  been  adopted  by  your  organization  to  express  your  views  about  how  uncertain  you 
are  of  the  resulting  judgments. 

10.7  Finally 

The  last  step  in  assessment  is  to  compare  the  scope  and  content  of  your  assessment  to  the 
intelligence  issue  statement  you  created  earlier.  Do  you  have  the  judgments  necessary  to  address 
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the  intelligence  issue?  If  not,  why  not?  What  is  required  to  produce  any  additional  judgments 
that  might  be  necessary? 

The  expectation  is  that  any  deficiencies  identified  at  this  stage  require  only  some  re-examination 
of  the  assignment  of  meaning  or  subsequent  phases.  However,  as  mentioned  earlier,  the 
possibility  exists  that  doing  the  analysis  has  resulted  in  much  greater  and  different  appreciation 
of  the  intelligence  issue.  If  so,  you  might  have  to  start  with  a  new  approach  to  information 
acquisition.  During  this  new  information  acquisition,  you  will  add  any  new  relevant  information 
that  results  to  the  prior  relevant  information.  You  must  then  repeat  the  FAC  approaches  for 
assigning  meaning,  developing  an  overall  belief,  and  making  your  prediction  to  complete  your 
efforts. 

We  suggest  that  you  capture  in  writing  the  process  and  results  of  forming  predictions.  The 
written  representation  should  include  a  description  of  the  analogs  used  in  aiding  the  predictions, 
any  special  conditions  associated  with  the  target  that  caused  you  to  make  adjustments  and  all 
milestones  used. 


181 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 
88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


11.0  ANALYSIS  REPRESENTATION 


11.1  Introduction 

You  have  now  completed  all  of  the  essential  elements  of  analysis  that  we  recommend  in  the 
FAC.  For  each  element  you  were  asked  to  capture  your  thinking  in  writing  because  of  the 
important  role  writing  plays  in  stimulating  your  cognition.  Now  is  the  time  to  integrate  all  of 
these  written  representations  together  to  fonn  a  comprehensive  and  coherent  analysis 
representation.  This  analysis  representation  represents  the  totality  of  your  analytical  efforts. 
Before  you  start,  however,  we  recommend  that  you  reread  the  “Writing  as  Thinking”  subsection 
of  the  “Analysts  as  Individuals”  section  of  the  FAC. 

As  we  describe  in  “Writing  as  Thinking,”  preparation  of  the  analysis  representation  is  beneficial 
in  three  ways: 

•  Writing  can  be  used  not  only  to  communicate  with  others,  but  also  to  help  you 
generate  ideas,  to  engage  in  self-reflection  as  you  develop  your  thoughts  about  a 
topic,  and  to  connect  and  develop  those  ideas  into  coherent  themes  and  assessments. 

•  Once  you  have  completed  it,  the  analysis  representation  allows  you  to  change  your 
role  from  writer  to  critic.  You  can  examine  what  you  have  written  thoroughly  and 
ask  yourself  “why  did  I  say  what  I  said?”  You  can  think  about  the  thinking  that  the 
written  word  represents  and  take  a  closer  look  at  your  evidence,  assumptions, 
beliefs,  judgments,  and  conclusions. 

•  Just  as  the  analysis  representation  provides  a  source  for  self-criticism,  it  also  serves 
as  the  basis  for  you  to  provide  answers  to  others  who  would  be  critical  of  your 
output.  Any  discussion  can  then  focus  on  not  just  the  judgments  or  conclusions  but 
on  the  cognition  documented  in  the  analysis  representation. 

You  write  the  analysis  representation  for  your  own  use.  You  are  both  the  author  and  the 
customer.  You  have  no  constraints  with  regards  to  size,  fonnat,  or  style.  While  clear  and 
effective  writing  is  always  preferred,  the  emphasis  of  this  writing  is  to  include  the  totality  of 
what  you  did  and  write  it  out  in  a  coherent  manner. 

11.2  How  to  Begin 

During  your  analysis  you  have  written  material  associated  with  each  of  the  following  sections  of 
the  FAC: 


•  Intelligence  Issue  Statement 

•  Analysis  Plan 

•  Information  Plan 

•  Information  Acquisition 

•  Assignment  of  Meaning 

•  Assessment 

As  we  discussed  in  the  “Assessment”  section,  you  are,  cognitively,  a  different  person  than  the 
one  that  embarked  on  this  analytical  effort.  You  now  know  more  about  the  issue  under  analysis, 
but  the  process  of  knowing  more  has  not  been  a  linear  one.  Iterations  have  been  a  common 
occurrence  as  you  progressed  through  the  FAC  approaches.  In  many  cases,  what  you  discovered 
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required  a  return  to  an  earlier  stage.  Perhaps  you  did  not  capture  all  of  these  iterations  in  your 
written  representations.  As  a  result,  you  should  examine  those  written  representations  and 
modify  them  if  necessary  to  ensure  that  they  reflect  all  that  you  considered  or  accomplished. 

The  following  sections  contain  suggestions  for  the  content  from  previous  FAC  sections  that  you 
should  consider  including  in  the  analysis  representation. 

11.3  Content  of  the  Analysis  Representation 

11.3.1.  Intelligence  Issue  Statement 

Include  the  originally  agreed  upon  intelligence  issue  statement.  Update  both  the  target  view 
guide  and  the  customer  view  guide.  Describe  any  changes  that  might  be  required  either  in  terms 
of  neglected  considerations  or  unduly  placed  emphasis  on  other  considerations.  Explain  why 
these  changes  are  necessary.  Is  this  issue  more  or  less  pressing  now  than  when  the  analysis 
started?  If  so,  why?  If  changes  have  occurred,  rewrite  the  intelligence  issue  statement  and 
include  the  revised  issue  statement,  as  well  as  the  original  one.  In  your  written  description,  be 
explicit  as  to  whether  the  subsequent  analysis  is  based  on  the  original  issue  statement  or  the 
revised  one. 

11.3.2.  Analysis  Plan 

Include  the  original  analysis  plan.  If  the  issue  statement  has  changed,  rewrite  the  analysis  plan 
and  include  it.  Compare  the  rewritten  analysis  plan  with  what  was  accomplished.  If  the 
substantive  areas  changed,  indicate  whether  these  changes  resulted  from  subsequent  information 
acquisition,  analysis,  or  issue  changes.  For  those  areas  where  others  were  to  provide  an  input, 
did  you  receive  those  inputs?  If  not,  what  was  done?  Did  you  ignore  those  areas  when  doing  the 
assessment,  did  you  seek  input  elsewhere,  or  did  you  provide  the  needed  judgments  even  though 
they  were  outside  your  area  of  analytic  responsibility?  If  you  did  the  analysis  yourself,  what  was 
the  basis  for  your  judgments?  Was  the  schedule  met?  If  not,  when  were  things  accomplished? 
Why  did  the  schedule  change?  Were  you  provided  the  needed  resources?  If  not,  what 
adjustments  were  made  in  the  execution  of  the  analysis  effort?  Rewrite  the  analysis  plan  to 
reflect  what  you  actually  did  and  include  it  in  the  analysis  representation. 

11.3.3.  Information  Plan 

Include  the  original  information  plan.  If  the  broad  topics  areas  were  changed  in  the  analysis  plan, 
rewrite  the  information  plan  and  include  it.  Were  all  the  topic  areas  searched?  If  not,  why  not? 
Were  all  the  sources  used,  if  not  why  not?  If  others  were  to  acquire  some  of  the  needed 
information  for  you,  did  they  acquire  it?  If  not,  what  was  done  (if  anything)  to  compensate  for 
this  lack  of  information?  Were  the  schedules  met?  If  not,  what  was  the  effect  on  the  acquisition 
of  information?  Rewrite  the  information  plan  to  reflect  what  you  did  and  include  it. 

11.3.4.  Information  Acquisition 

From  your  search  diary  provide,  if  you  can,  the  following  infonnation: 

•  For  each  IS&R  system,  search  topic,  and  search  logic,  provide  data  on  the  number 
of  documents  retrieved  and  the  number  you  detennined  were  relevant. 

•  The  IS&R  system  source(s)  for  each  relevant  document. 

•  The  language  of  each  relevant  document. 
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•  For  each  search  topic,  include  the  search  logic  you  were  using  at  the  time  searching 

ceased  and  the  reason  why  you  stopped  searching. 

•  A  bibliographic  listing  of  all  relevant  documents. 

•  A  listing  of  all  people,  places,  and  things  found  in  relevant  information. 

11.3.5.  Assignment  of  Meaning 

From  the  written  material  you  created  while  assigning  meaning,  provide  the  following 
information: 

•  A  listing  of  all  the  people,  places  and  things  that  are  directly  or  indirectly  linked  to 

the  people,  places,  and  things  found  in  relevant  information. 

•  For  each  person  on  the  above  list,  provide  the  following  infonnation: 

>  A  bibliographic  listing  of  all  documents  containing  mention  of  the  person. 

>  A  listing  of  all  persons  with  whom  the  individual  has  been  associated  and 
the  time  period  of  the  association. 

>  A  listing  of  all  groups  or  organizations  with  whom  the  individual  has  been 
associated  and  the  time  period  of  the  association. 

>  The  specialties  and  substantive  activities  of  the  person  as  a  function  of  time. 

>  Your  belief/judgment  as  to  what  motivates  this  person  and  his  or  her  actions 
and  the  basis  for  your  belief/judgment. 

•  For  each  place  (organization/group)  on  the  above  list,  provide  the  following 

infonnation: 

>  A  bibliographic  listing  of  all  the  documents  containing  mention  of  the  place. 

>  A  listing  of  all  persons  that  have  been  associated  with  the  place  and  the  time 

period  of  the  association. 

>  The  specialties  and  substantive  activities  associated  with  the  place  as  a 
function  of  time. 

>  Your  belief/judgment  as  to  what  motivates  the  activities  or  behavior 
associated  with  the  place  and  the  basis  for  your  belief/judgment. 

•  For  each  thing  on  the  above  list,  provide  the  following  information: 

>  A  bibliographic  listing  of  all  documents  containing  mention  of  the  thing. 

>  A  listing  of  all  persons  and  places  that  have  been  associated  with  the  thing 

and  the  time  period  of  the  association. 

>  The  place  where  the  thing  was  developed  or  produced  as  a  function  of  time. 

>  The  people  who  designed/developed  the  thing  and  the  time  in  which  the 
design  or  development  was  accomplished. 

>  All  uses  made  of  the  thing. 

>  Your  belief/judgment  as  to  what  motivates  the  use  of  the  thing  and  the  basis 
for  your  belief/judgment. 

•  For  each  document  containing  relevant  infonnation,  provide  the  following: 

>  A  brief  summary  of  the  substantive  content  of  document. 
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>  A  comparison  of  how  the  content  of  this  document  differs  from  earlier 
documents  that  dealt  with  the  same  substantive  topic. 

>  Anything  new  or  different  in  the  substantive  content. 

>  The  relationship  between  the  document  content  and  the  intelligence  issue. 

>  Your  belief/judgment  on  the  contribution  of  the  document’s  content  to  the 
understanding  of  the  topic  or  capability  and  the  basis  for  your 
belief/judgment. 

>  A  listing  of  the  previously  determined  motivation  of  each  person,  place  or 
thing  associated  with  the  information. 

>  Your  belief/judgment  on  the  meaning  of  the  information  and  the  basis  for 
your  belief/judgment. 

11.3.6.  Assessment 

From  the  written  material  you  did  during  your  assessment,  provide  the  following  information: 

•  A  listing  of  the  meaning  of  each  piece  of  relevant  information  sorted  by  the 
date  associated  with  the  information. 

•  Listings  sorted  by  content,  places,  or  persons  developed  from  the 
commonality  of  the  meaning  of  each  piece  of  relevant  information 
considered. 

•  A  listing  of  beliefs/judgments  provided  from  other  areas  of  analytical 
responsibility. 

•  A  listing  of  intermediate  beliefs/judgments  and  their  basis. 

•  A  listing  of  final  beliefs/judgment(s)  and  their  basis. 

•  A  chronological  listing  of  known  events  and  activities  that  relate  to  the 
intelligence  issue. 

•  A  description  of  any  predictions  made  and  the  basis  for  the  predictions. 

•  A  description  of  your  uncertainties  for  both  the  final  judgment  and  any 
predictions. 

•  The  rationale  associated  with  each  of  your  uncertainties  associated  with  the 
final  judgment  and  any  predictions. 

11.4  Changing  from  Writer  to  Critic 

When  you’ve  accomplished  all  of  the  above,  your  role  changes  from  that  of  a  writer  to  that  of  a 
critic.  The  overall  FAC  approaches  are  based  on  work  from  the  “bottom  up.”  However,  in  your 
role  as  a  critic,  you  want  to  work  from  the  “top  down.”  You  can  start  with  either  the  predictions 
made  or  the  final  belief(s)  and  ask  yourself  why.  Each  answer  to  a  why  question  must  also  be 
subjected  to  a  why  question  until  there  are  no  more  questions  to  be  answered. 

Your  objectives  in  such  intense  questioning  of  yourself  are  twofold:  to  discover  any  flaws  in 
your  bottom  up  analysis  and  belief  formulation  and  to  identify  the  beliefs  that  are  best  supported 
by  evidence  and  have  the  greatest  contribution  to  supporting  other  beliefs.  In  effect,  you  are 
marshalling  your  arguments  for  what  you  believe. 
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You  will  find  this  marshalling  to  be  extremely  useful  in  the  preparation  of  the  customer  product. 
Your  preparation  of  the  customer  product  is  the  last  step  in  the  FAC  and  fulfills  the  ultimate 
objective  of  addressing  the  intelligence  issue  and  providing  your  assessment  to  customers  that 
need  the  information. 

11.5  Summary 

The  totality  of  the  written  material  that  you  have  prepared  in  this  section  is  the  analysis 
representation.  Writing  out,  in  a  thorough  and  coherent  manner,  what  you  did  throughout  the 
analysis  provides  a  sound  foundation  for  preparing  product(s)  for  your  customer(s).  This  final 
step  in  the  FAC  is  described  in  the  next  section. 
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12.0  CUSTOMER  PRODUCT 


“A  final  consequence  of  the  lack  of  realization  of  the  impact  of  beliefs. ...is  that  actors  often  fail 
to  maximize  their  ability  to  communicate. ..what  matters  in  sending  a  message  is  not  how  you 
would  understand  it,  but  how  others  will  understand  it. ...To  put  oneself  in  another ’s  skin  is 

terribly  hard.  ”I9<> 

12.1  The  Last  Cognitive  Challenge 

In  earlier  sections  of  the  FAC  we  described  how  you  can  conquer  the  cognitive  challenges  you 
encounter  during  intelligence  analysis.  This  section  describes  how  to  address  the  final  cognitive 
challenge  in  your  analysis  journey:  communicating  your  assessment  to  a  customer  or  audience. 

As  we  described  in  the  FAC  section  on  intelligence  issues,  you  got  started  on  your  analysis  in 
one  of  three  ways:  you  were  assigned  a  new  AOR  (AOR  issue);  a  question  or  issue  was  initiated 
by  you  (self-initiated  issue);  or  someone  else  initiated  a  question  for  you  to  address  (external 
issue).  The  FAC  led  you  through  the  process  of  understanding  that  issue  more  thoroughly, 
planning  for  analysis  and  information  retrieval,  gathering  information,  assessing  infonnation, 
and  formulating  your  assessment  on  the  intelligence  issue.  For  AOR  issues,  the  FAC  was  used 
to  understand  the  scope  of  your  responsibilities,  identify  the  topics  or  functional  areas  associated 
with  your  AOR,  and  develop  a  plan  to  acquire  information  on  those  topics  or  functional  areas.  As 
you  do  analysis  within  your  AOR,  you  identify  self-initiated  issues  or  others  will  generate 
external  issues  that  require  a  customer  product.  This  section  applies  to  self-initiated  and  external 
issues.  For  self-initiated  and  external  issues,  it  is  now  time  to  share  your  analysis  with  those  that 
can  use  it  to  do  their  missions. 

Intelligence  analysts  are  expected  to  communicate  their  assessments  in  a  wide  variety  of  fonnats 
using  a  variety  of  presentation  methods.  No  matter  what  communication  method  or  medium  you 
use,  you  can  use  the  work  that  you  have  done  throughout  the  FAC  to  prepare  products  to 
communicate  to  various  audiences.  The  purpose  of  this  section  is  to  aid  you  in  developing  an 
effective  communication  of  your  assessment.  The  section  focuses  on  developing  a  written 
communication  or  product  because  cognitively  the  written  product  will  be  the  most  challenging 
product.  The  writing  you  do  can  then  serve  as  an  excellent  foundation  for  adapting  the  written 
product  to  other  forms  of  communication,  such  as  briefings  and  discussion  sessions. 

Who  is  an  audience? 

As  we  described  in  the  FAC  section  on  intelligence  issues,  you  have  various  customers  or 
audiences.  If  it  is  an  external  issue,  the  primary  audience  is  the  individual  or  organization  that 
initiated  the  issue.  For  other  types  of  issues,  as  we  have  described  in  other  FAC  sections,  your 
audience  will  be  prospective  customers  that  you  think  can  use  the  assessment  you  have  done. 
These  may  include  individuals  within  or  outside  of  intelligence  organizations  (including  your 
own),  as  well  as  organizational  audiences  both  within  the  intelligence  community  and  external  to 
it.  These  audiences  include  other  analysts  or  managers,  as  well  as  an  individuals  or 
organizations  that  will  use  your  assessment  for  decision  making  and  action  to  fulfill  their 
missions. 


190  Jervis,  R.  Perception  and  Misperception  in  International  Politics,  Princeton  University  Press,  Princeton,  NJ, 
1976.  p.  187. 


187 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 
88ABW/PA  cleared  15  Aug  12,  88ABW-2012-4458 


Why  do  you  regard  communicating  the  results  of  intelligence  analysis  to  an  audience  or 
customers  as  a  cognitive  challenge? 

In  the  FAC,  we  have  emphasized  that  information  does  not  have  a  single  meaning.  For  the 
customer  product,  the  recipient  that  will  detennine  meaning  is  your  audience.  Just  as  we 
discussed  the  role  of  your  mindware  in  detennining  meaning,  for  the  customer  product  it  is  the 
audience’s  mindware  that  will  be  used  to  detennine  the  meaning  of  what  you  provide.  As  you 
recall,  this  mindware  includes  both  content  (facts,  data,  information)  and  the  audience’s 
knowledge/beliefs  and  goals.  Your  purpose  and  goals  in  addressing  the  intelligence  issue  were 
instrumental  in  your  determination  of  meaning.  Similarly,  the  audience’s  purpose  will  govern 
their  determination  of  meaning. 

As  we  have  discussed  in  the  “Assignment  of  Meaning”  section,  as  an  intelligence  analyst,  you 
are  focused  on  determining  the  originator’s  (i.e.,  your  target’s)  meaning.  For  your  assessment, 
you  are  the  originator  of  the  information  to  be  communicated  to  the  audience.  As  we  pointed 
out,  focusing  on  the  originator’s  meaning  is  unique  to  intelligence  analysis.  Instead,  your 
audience  will  be  primarily  focused  on  what  the  information  means  to  them  and  how  they  can 
potentially  use  it,  i.e.,  how  does  it  help  them  do  the  mission  that  they  have  to  do.  Consequently, 
your  challenge  is  to  achieve  shared  meaning  with  the  audience.  You  have  formed  your  beliefs 
about  the  target  and  now  you  want  to  establish  a  set  of  shared  beliefs  about  that  target  with  your 
audience.  Because  you  and  the  audience  have  different  mindware,  it  will  be  a  major  cognitive 
challenge  for  you  to  determine  what  is  necessary  to  attain  shared  meaning  and  beliefs  with  your 
audience. 

We’ve  said  that  the  purpose  of  this  section  is  to  aid  you  in  developing  an  effective 
communication  of  your  assessment.  Stated  differently,  the  purpose  is  to  aid  you  in  attaining  a 
shared  meaning  of  your  assessment  with  your  audience.  If  you  are  entirely  successful,  after 
reading  your  product  you  and  the  customer  will  share  the  beliefs  that  you  fonned  about  the 
target;  the  customer  will  use  those  beliefs  to  accomplish  their  mission. 

All  of  the  work  you  do  to  create  the  customer  product  must  be  oriented  to  the  needs  and  role  of 
the  audience.  This  can  present  a  challenge  because  you  are  writing  for  an  audience  whose 
environment,  relationship  to  the  subject  of  the  intelligence  issue,  and  relationship  to  you  must  all 
be  taken  into  consideration  as  you  write.  In  addition,  your  mindware  most  likely  represents  more 
facts,  details,  context,  and  considerations  than  the  customer  needs  to  know.  Incorporating  all  of 
these  perspectives  into  your  writing  of  the  customer  product  will  require  a  change  of  perspective 
on  your  part.  Developing  a  more  complete  understanding  of  your  audience’s  perspective  and 
keeping  that  understanding  at  the  forefront  of  your  thinking  while  you  create  the  product  is  an 
ongoing  cognitive  challenge. 

12.2  What  We  Don’t  Cover  in  This  Section 

We’d  like  to  mention  some  topics  that  won’t  be  discussed  in  this  section  of  the  FAC:  format, 
style,  grammar,  organizational  and  intelligence  community  requirements  and  standards  for 
products,  and  organizational  review  processes.  We  suggest  that  you  familiarize  yourself  with 
your  organization’s  policies  and  requirements  for  document  layout  and  content  so  that  you  can 
produce  a  document  that  meets  your  organization’s  requirements.  You  should  also  seek 
information  within  your  organization  about  standards,  including  those  of  your  organization  and 
the  intelligence  community.  In  addition,  you  should  also  be  familiar  with  your  organization’s 
product  review  process. 
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While  we  won’t  cover  reviews  in  this  FAC  section,  an  important  point  to  keep  in  mind  was 
addressed  in  the  “Analysts  as  Individuals”  and  “Knowing  Yourself,”  section:  instinctively  we 
may  not  readily  accept  critical  comments  on  our  efforts.  However,  research  suggests  that  we 
shouldn’t  immediately  reject  comments  since  we  may  be  rejecting  a  learning  experience.  Others 
may  have  more  extensive  contacts  and  expertise,  as  well  as  a  fresh  perspective. 

As  for  style  and  grammar,  a  multitude  of  books  and  online  tools  exist  which  cover  these  two 
topics,  some  of  which  are  specifically  oriented  to  intelligence  analysis191.  In  this  section  of  the 
FAC,  we  assume  that  you  have  a  working  knowledge  of  the  basics  of  style,  grammar,  and 
composition.  If  you  struggle  with  writing,  or  would  like  to  improve  your  writing  skills,  we 
suggest  that  you  investigate  the  many  resources  available  through  your  organization,  local 
college  or  university,  or  library. 

12.3  How  Will  I  Use  the  Analysis  Representation  to  Prepare  a  Customer  Product? 

You  wrote  your  analysis  representation  for  your  own  use;  you  were  both  the  author  and  the 
customer.  The  analysis  representation  was  not  constrained  by  size,  format  or  style.  The  focus 
was  to  include  the  totality  of  what  you  did  in  the  entire  analysis  process  and  write  it  out  in  a 
coherent  manner.  It  helped  you  pull  together  all  of  the  written  materials  you  prepared  during 
each  step  of  the  FAC.  While  preparing  the  analysis  representation,  you  may  have  also  written 
more  information  to  fully  capture  your  thinking  throughout  the  analysis  process. 

To  do  the  customer  product,  you’ll  use  the  written  material  you  updated  and  wrote  for  the 
analysis  representation  section.  For  example,  the  issue  statement  and  customer  and  target  view 
guides  will  help  you  analyze  your  audience  for  the  customer  product.  For  the  content  of  your 
customer  product,  you  will  rely  on  the  writing  you  did  for  the  assessment  section.  However,  you 
will  need  to  reframe  your  assessment  to  communicate  it  based  on  the  analysis  of  your  audience’s 
needs  that  you  will  do  as  part  of  this  section.  That  refraining  will  most  likely  involve  revising 
the  writing  that  you’ve  already  done.  In  the  customer  product,  you  will  also  use  the  information 
about  the  uncertainty  associated  with  your  assessment;  you  will  need  to  make  decisions  about 
what  and  how  to  communicate  that  uncertainty  to  the  customer.  Unlike  the  analysis 
representation,  you  will  have  constraints  on  size,  format,  and  style  for  the  customer  product  that 
will  also  influence  the  manner  in  which  you  reframe  your  assessment  for  the  audience. 

12.4  Audience  Analysis:  Determining  What  the  Customer  Needs 

Before  you  begin  creating  the  customer  product  you  must  undertake  a  critical  activity:  an 
audience  analysis.  This  analysis  involves  understanding  both  the  implied  and  expressed 
characteristics  of  the  audience  for  the  product  you  are  about  to  create.  You  must  understand  both 
the  implied  and  expressed  characteristics  of  the  audience  to  write  an  effective  customer  product. 
Earlier  sections  of  the  FAC  explored  the  characteristics  of  the  customer  and  what  they  needed  to 
know,  but  a  fuller  understanding  of  the  customer  will  assist  you  in  creating  a  more  effective 
product. 

What  should  I  consider  about  implied  characteristics? 

Implied  characteristics  are  those  which  are  likely  to  be  part  of  the  culture  of  your  intelligence 
organization  and  its  overall  understanding  of  its  customers.  These  characteristics  may  be 


191  For  example,  Major,  J.  S.  Writing  Classified  and  Unclassified  Papers  for  National  Security.  Scarecrow  Press, 
Lanham,  MD,  2009. 
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unexpressed  but  widely  accepted.  For  example,  consider  the  intelligence  analysis  literature  and 
how  it  describes  some  customers.  The  intelligence  literature  often  characterizes  intelligence 
customers  in  the  following  ways: 

•  The  customer  uses  the  results  of  intelligence  analysis  to  make  decisions,  direct 
military  operations,  or  develop  policy. 

•  The  customer  may  know,  or  believes  he  or  she  knows,  more  about  a  topic  than  the 
intelligence  analyst. 

•  The  customer  is  only  interested  in  the  bottom  line. 

•  The  customer  may  or  may  not  read  the  detailed  argument  you  develop  to  justify 
your  conclusions. 

•  Some  customers  are  interested  in  details;  other  customers  only  care  about  the  big 
picture  of  the  issue. 

•  The  customer  is  operating  under  a  set  of  pressures  or  deadlines  that  are  different 
than  those  of  intelligence  analysts. 

These  characterizations  reflect  beliefs  about  the  nature  of  the  audience  for  the  customer  product. 

Characterizations  of  the  customer,  however,  may  not  always  be  accurate.  In  order  for  you  to 
create  an  effective  customer  product,  you  need  to  consider  whether  or  not  the  characterizations 
are  true  for  your  audience.  If  not,  what  are  some  of  the  ways  that  your  organization  views  its 
customers?  Is  your  audience  other  intelligence  analysts  rather  than  decision  makers?  You  should 
think  carefully  about  broad  organizational  assumptions  about  the  needs  of  the  customer  or 
audience  before  you  write  the  customer  product. 

What  expressed  characteristics  should  I  consider? 

In  addition  to  these  implied  general  characterizations,  your  audience  has  some  more  specific  and 
unique  qualities  as  well.  These  are  expressed  characteristics  of  the  customer,  some  of  which  you 
captured  earlier  in  the  FAC  when  you  initially  wrote  the  intelligence  issue  and  then  updated  it  in 
the  analysis  representation.  The  Customer  View  Guide  helped  you  understand  as  much  as 
possible  about  why  a  customer  was  asking  the  intelligence  question  and  how  he  or  she  planned  to 
use  your  assessment  (i.e.,  the  customer’s  purpose,  roles,  and  missions).  For  self-initiated  issues, 
you  explored  how  prospective  customers  would  use  your  assessment.  The  Target  View  Guide 
also  helped  you  understand  what  the  actual  or  prospective  customers  need  to  know  about  the 
target.  The  understanding  you  achieved  from  doing  these  two  guides  was  then  incorporated  into 
the  Intelligence  Issue  Statement.  These  three  documents  contain  very  useful  information  for 
your  audience  analysis.  Your  understanding  of  the  specific  needs  of  an  actual  or  prospective 
customer  has  driven  all  of  your  activities  in  the  FAC  and  must  be  part  of  your  framing  and 
writing  of  the  customer  product. 

In  addition,  the  “Assignment  of  Meaning”  section  described  three  concepts  that  are  essential  for 
analyzing  your  target:  meaning ,  habits,  and  structure.  From  a  different  perspective,  these 
concepts  are  also  useful  for  analyzing  your  audience.  Earlier  in  this  section,  we  described  the 
cognitive  challenge  associated  with  sharing  your  meaning  with  the  recipient  of  the  assessment 
(your  audience)  since  they  have  different  mindware.  The  second  two  concepts,  habits  and 
structure,  are  also  useful  in  analyzing  your  audience. 
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Recall  that  it  is  important  to  understand  a  person’s  habits  and  the  context  in  which  they  occur.  It 
is  also  important  to  be  aware  when  that  context  has  changed  and  to  identify  what  is  unknown 
about  the  context.  Similarly,  analyzing  the  habits  and  context  of  your  audience  can  provide 
useful  insights  for  your  framing  of  the  customer  product.  Your  audience  also  operates  within  a 
structure  and  all  decisions  and  actions  take  place  within  that  structure.  Analyzing  that  structure 
can  provide  additional  understanding  that  will  help  you  effectively  communicate  to  the  audience. 

By  coupling  your  understanding  of  the  audience’s  mindware,  habits,  and  structure  from  prior 
FAC  work  with  additional  audience  analysis,  you  can  gain  additional  insights  about  the 
customer.  This  will  aid  you  in  focusing  on  your  purpose  in  this  section:  using  your 
understanding  of  the  audience  to  frame  and  communicate  your  assessment.  You  will  likely  find 
that  you  need  to  analyze  other  factors  to  better  understand  your  audience’s  purpose  and 
characteristics.  These  factors  include: 

•  The  environment  of  the  audience 

•  The  subject  of  the  intelligence  issue  as  interpreted  by  the  audience 

1 92 

•  The  relationship  of  the  audience  and  the  writer 

Some  additional  questions  to  better  understand  these  three  factors  include: 

•  What  is  the  audience’s  experience  and  educational  background? 

•  How  much  does  the  audience  know  about  the  overall  subject,  about  the  specific 
issue,  about  what  I  intend  to  say?  How  strong  are  the  audience’s  beliefs  about  the 
subject  and  the  issue? 

•  Has  the  audience  acted  on  those  beliefs  before?  Why  does  the  audience  act/react  the 
way  they  do? 

•  Why  does  the  audience  have  these  beliefs?  Does  the  audience  have  beliefs  which 
are  different  than  those  you  hold  about  the  subject  or  issue? 

•  What  is  the  history  of  the  audience  with  this  or  similar  issues?  Is  this  a  new  issue 
for  the  audience? 

•  Has  the  audience  provided  input  or  feedback  before  on  this  or  similar  issues?  What 
was  the  nature  of  it? 

•  What  is  the  history  of  the  audience  with  my  organization?  With  the  intelligence 
community? 

•  What  is  the  audience’s  knowledge  and  attitude  about  my  organization?  About  those 
within  my  organization  that  have  had  contact  with  the  audience? 

•  Do  I  or  those  in  my  organization  share  experiences,  attitudes,  subject  matter 
interests  with  the  audience?  What  interests  are  shared,  what  are  different?  Do  we 
have  a  shared  purpose? 

•  Given  the  audience’s  characteristics,  is  this  an  appropriate  audience  for  this  subject? 
If  so,  what  framing,  supporting  argument  presentation,  and  tone  will  likely  be 
effective  in  sharing  meaning? 


192  Factors  and  questions  adapted  from  Pfister,  F.R.  &  Petrick,  J.F.  "A  Heuristic  Model  for  Creating  a  Writer's 
Audience,"  College  Composition  and  Communication,  Vol.  31,  No.  2,  1980,  p.  214. 
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As  you  go  through  these  questions,  you  will  undoubtedly  think  of  other  questions  associated  with 
your  audience.  The  result  of  your  audience  analysis  should  be  a  vision  of  the  person  or 
organization  that  you  are  communicating  with  before  you  begin  writing.  We  recommend  you 
capture  your  audience  analysis  in  writing  (in  a  fonn  of  your  choice,  e.g.,  a  description  or  a  list  of 
characteristics)  in  preparation  for  the  next  step  in  creating  the  customer  product. 

The  results  of  my  intelligence  analysis  work  are  to  be  used  by  an  organization,  not  a  specific 
person.  How  does  your  advice  apply  to  me? 

Organizations  have  implied  and  expressed  characteristics  just  like  those  of  individuals.  When  we 
talk  about  the  audience,  we  mean  either  an  individual  or  an  organization.  You  can  apply  the 
audience  analysis  methods  we  describe  to  both  organizations  and  individuals  alike. 

12.5  Getting  Ready  to  Write 

Once  you  have  completed  your  audience  analysis,  you  must  carefully  consider  what  you  plan  to 
include  in  the  customer  product.  Given  your  audience  analysis,  what  content  will  be  necessary  to 
both  infonn  the  audience  and  persuade  that  audience  to  share  your  beliefs  about  the  target?  You 
must  decide  what  broad  topical  categories  to  include,  but  that  is  not  sufficient.  In  addition,  you 
must  decide  what  specific,  focused  topics  to  include  and  how  to  construct  them  with  precise, 
clear  language.  If  your  product  contains  too  little  information  of  substantive  value,  or  if  your 
judgments  are  not  adequately  supported,  the  customer  is  likely  to  dismiss  your  judgments.  If 
your  product  contains  too  much  infonnation,  the  customer  may  get  frustrated  at  your  lack  of 
focus  and  dismiss  the  judgments  that  you  are  trying  to  convey.  Neither  of  these  outcomes  is 
desirable. 

If  you  are  having  difficulty  determining  what  to  include  in  the  customer  product,  it  may  be 
helpful  to  consider  again  what  your  purpose  is  in  creating  this  communication  to  the  customer. 
What  is  your  reason  for  creating  a  customer  product?  How  do  you  want  the  audience  to  respond? 
What  decisions  and  actions  does  your  analysis  support?  A  review  of  your  thinking  about  these 
questions  and  your  audience  analysis  should  help  you  select  and  reframe  the  appropriate 
information  about  your  assessment  for  the  product. 

12.6  Components  of  the  Customer  Product 

The  two  major  components  of  the  customer  product  are: 

•  A  concise  assessment  of  the  intelligence  issue 

•  A  narrative  of  the  assessment  of  the  intelligence  issue 
We’ll  describe  each  of  these  components  in  more  detail  in  the  following. 

12.6.1.  Concise  Assessment 

If  you  happened  to  bump  into  the  customer  and  he  or  she  asked  you  what  you  thought  about  the 
intelligence  issue,  you  wouldn’t  spend  an  hour  explaining  what  you  have  concluded.  You  would 
be  more  likely  to  answer  the  customer’s  question  in  a  few  sentences  which  would  summarize 
your  assessment. 

The  concise  assessment  we  suggest  you  write  is  similar  to  what  you  would  say  in  a  brief 
encounter  with  the  customer:  it  is  a  summary  of  your  judgments.  The  concise  assessment  might 
also  include  a  few  words  that  describe  key  evidence  and  uncertainties  about  your  judgments.  The 
concise  assessment,  however,  does  not  include  the  argument  or  chain  of  reasoning  that  supports 
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your  judgments.  Besides  being  brief  and  to  the  point,  you  must  frame  the  concise  assessment  for 
your  audience  based  on  your  audience  analysis. 

We  recommend  that  you  begin  writing  the  product  by  drafting  the  concise  assessment.  What  are 
the  major  points  you  want  to  convey  to  the  audience?  This  will  help  focus  your  thinking  about 
how  to  frame  the  longer  narrative  assessment.  After  you’ve  drafted  and  reviewed  the  narrative 
assessment,  you’ll  want  to  revisit  and  adjust  your  concise  assessment.  You’ll  likely  find  that 
you’ll  iterate  several  times  between  the  two  to  refine  your  message  for  the  audience. 

Why  is  the  concise  assessment  important? 

An  effective  customer  product  informs  and  persuades  the  audience.  In  order  to  be  persuasive, 
every  element  of  the  customer  product  you  create  must  relate  to  the  main  idea  you  are  trying  to 
communicate.  If  you  can’t  identify  the  main  idea  contained  in  your  assessment  of  the  intelligence 
issue  and  write  it  in  a  way  that  the  customer  can  understand  it,  you  cannot  convey  your 
supporting  evidence  and  associated  argument  in  a  way  that  convinces  the  customer  to  share  your 
beliefs  about  the  target.  Your  narrative  assessment  will  lack  focus  and  you  will  fail  to  inform  and 
persuade  the  audience. 

Also,  it  may  be  difficult  for  you  to  change  your  perspective  on  the  intelligence  issue  from  your 
own  point  of  view  to  the  point  of  view  of  the  audience.  Writing  the  concise  assessment  is  a  good 
first  step  in  conquering  the  cognitive  challenge  of  writing  for  the  audience  rather  than  writing  for 
your  own  use. 

Finally,  it  is  quite  common  in  customer  products  to  offer  the  bottom  line  of  the  analysis  first  in 
the  document  or  presentation.  The  audience  wants  to  know  whether  to  read  (or  listen  to)  your 
product  or  skip  it.  The  concise  assessment  plays  an  important  role  in  capturing  the  audience’s 
attention  and  setting  the  stage  for  a  more  detailed  discussion  of  the  intelligence  issue  in  the 
assessment  narrative. 

12.6.2.  Assessment  Narrative 

Writing  your  assessment  out  as  a  story  specifically  reframed  for  your  audience’s  perspective  is 
essential  to  attain  shared  meaning  of  your  judgments.  That  story  is  the  assessment  narrative. 

What  elements  does  the  assessment  narrative  include? 

The  most  important  thing  to  remember  about  the  narrative  is  that  everything  you  include  in  it 
must  contribute  to  your  purpose:  sharing  the  meaning  of  your  assessment  with  the  audience. 
Your  narrative  will  include  various  elements.  You’ll  need  to  provide  your  judgments  and  the 
necessary  support  for  those  judgments  so  that  the  audience  will  adopt  the  beliefs  you  have 
formed  about  the  intelligence  issue.  The  narrative  also  includes  aspects  of  your  argument  or 
chain  of  reasoning.  Arguments  consist  of  evidence,  hypotheses,  and  statements  that  justify 
linkages  between  evidence  and  hypotheses.  Your  argument  is  your  chain  of  reasoning  from 
evidence  to  hypotheses.  It  includes  the  assumptions  you  made  in  your  analysis,  as  well  as 
judgments  of  others  that  you  relied  upon  for  your  assessment  in  the  narrative.  You  developed 
your  assumptions,  evidence,  and  tentative  hypotheses  during  your  analysis  in  the  Assignment  of 
Meaning  and  Assessment  sections.  Your  analysis  resulted  in  beliefs  at  various  stages  (i.e., 
beginning  and  intermediate  beliefs)  and  ultimately  the  overall  belief  that  forms  the  basis  for  the 


193  Anderson,  T.,  Schum,  D.A.  &  Twining,  W.L.  Analysis  of  Evidence,  2nd  edn,  Cambridge  University  Press, 
Cambridge,  UK,  2005,  p  60. 
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judgments  and  conclusions  that  are  your  assessment.  The  narrative  organizes  your  assessment 
into  a  coherent  story  that  is  logically  supported  by  the  evidence,194  presented  as  a  meaningful 
whole  or  totality.195 

The  narrative  includes  an  explanation  of  the  current  status  of  the  target  and  your  prediction  about 
the  future  state  of  the  target’s  activities,  developments,  and  intent.  To  convey  your  assessment, 
your  narrative  includes  selected  descriptions  of  people,  places,  things,  decisions,  actions,  and 
time  frames  associated  with  the  target,  as  situated  in  the  target’s  environment.  It  is  also  be 
necessary  to  include  appropriate  background  and  context  for  the  infonnation  you  choose  to 
include  in  your  narrative.  Based  on  your  audience  analysis,  all  of  these  elements  must  be  framed 
relative  to  the  interests  and  beliefs  of  your  audience. 

What  should  I  consider  including  for  various  elements  of  the  narrative? 

What  you  include  relative  to  the  narrative  elements  described  in  the  preceding  depends  on  many 
factors  in  addition  to  the  interests  and  purpose  of  your  audience;  for  example,  the  subject  matter 
associated  with  the  intelligence  issue,  your  purpose,  and  the  nature  of  your  evidence  and 
judgments.  Consequently,  we  can’t  advise  you  as  to  exactly  what  you  should  include  or 
precisely  how  to  organize  your  narrative.  The  following  descriptions,  however,  provide 
recommendations  about  what  you  should  consider  relative  to  selected  narrative  elements. 
Appendix  1  also  contains  questions  to  aid  you  in  your  review  of  your  completed  narrative.  Keep 
in  mind  that  the  elements  of  the  narrative  will  be  interwoven  to  tell  your  story;  your  writing  will 
not  be  segmented  by  element. 

•  Beliefs:  In  the  Assessment  section,  you  developed  your  overall  belief(s).  Your 
analysis  may  have  resulted  in  a  single  belief  or  multiple  beliefs  about  the  target 
based  on  what  was  best  supported  by  the  evidence.  If  your  analysis  resulted  in 
multiple  beliefs,  it  is  appropriate  to  convey  that  to  your  audience.  How  you 
communicate  multiple  beliefs,  often  referred  to  as  alternative  hypotheses  in  the 
intelligence  literature,  depends  on  how  the  audience  may  use  the  infonnation  you 
provide,  as  well  as  the  strength  of  each  of  those  beliefs  based  on  the  evidence.  The 
audience,  however,  may  want  a  “single  answer”.  In  this  case  you  should  provide 
your  strongest  belief,  with  associated  uncertainties,  as  your  assessment  or  “best 
answer”  to  the  customer.  If  you  think  that  the  customer  needs  to  know  about  other 
possible  outcomes  given  how  they  could  use  your  assessment,  you  should  also 
provide  those  even  if  a  customer  hasn’t  asked  for  them  or  wants  just  a  “single 
answer.” 

•  Evidence:  Your  beliefs  rest  on  the  evidence  that  you  acquired.  All  of  the 
significant  evidence  that  provides  the  foundation  for  your  beliefs  and  resulting 
assessment  should  be  accounted  for  in  your  narrative.196  Evidence  is  significant  if  it 
was  instrumental  in  fonning  the  beliefs  that  resulted  in  your  final  assessment.  You 
should  also  include  any  evidence  of  consequence  which  does  not  support  your 
assessment.  It  is  also  essential  that  your  narrative  address  questions  about  the 


194  Anderson  p.  152. 

195  Anderson,  p.  155. 

196Schum,  D.A.  Narratives  in  Intelligence  Analysis:  Necessary’  but  often  Dangerous,  George  Mason  University, 
Fairfax,  VA,  2005,  p.8. 
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credibility,  relevance  and  force  of  the  evidence  you  present,197  including  any 
assumptions  or  speculations  you’ve  made  to  fill  in  gaps  in  the  narrative  left  by 
evidence  you  don’t  have.  The  strength  of  your  conclusion  depends  on  how  many 
recognized  questions  remain  that  are  unanswered  by  the  evidence  presently 
available.  Consequently,  it  is  important  to  recognize  the  completeness  of  your 
evidence.199  Given  your  assessment,  you  should  also  discuss  what  evidence  you 
expected  to  see  but  do  not.  It  may  be  appropriate  to  explain  why  you  think  this  is 
the  case  (e.g.,  not  collectable  because  of  lack  of  coverage  or  the  nature  of  the 
evidence).  All  evidence  must  be  appropriately  sourced. 

•  Assumptions:  Keep  in  mind  the  FAC  definition  of  assumptions— statements  or 
descriptions  that  are  treated  as  if  they  are  true  without  the  necessity  of  proving  them 
true.  Assumptions  are  often  used  to  establish  boundary  conditions  for  subsequent 
analysis.  The  judgments  and  assessments  of  other  analysts  used  in  your  assessment 
are  not  assumptions  unless  they  are  used  to  establish  boundary  conditions  in  a 
“what  if’  analysis.  The  substantive  areas  you  identified  in  your  analysis  plan  reflect 
your  assumptions  about  the  intelligence  issue  when  you  started  your  analysis.  These 
substantive  areas  were  used  to  determine  the  judgments  you  needed  from  others  for 
topics  outside  your  AOR.  If  you  were  unable  to  obtain  the  needed  judgments  from 
others,  you  had  to  establish  your  own  boundary  conditions  for  these  topics.  It  is 
important  to  tell  the  audience  about  those  boundary  conditions.  In  addition,  if  those 
boundary  conditions  were  critical  to  your  argument,  it  introduces  significant 
uncertainty  about  your  assessment.  This  uncertainty  must  be  conveyed  to  the 
audience  in  a  way  that  is  relevant  to  their  decisions  and  actions.  Clearly  identify  the 
boundary  conditions  or  assumptions  used  in  your  assessment  and  articulate  how 
critical  they  are  to  key  elements  of  your  argument.  Be  precise  about  describing  the 
boundary  conditions  or  assumptions  you  established  and  differentiating  them  from 
the  judgments  of  others. 

•  Judgments  of  others:  The  judgments  and  assessments  of  other  analysts  used  in 
your  assessment  should  be  clearly  presented  as  other’s  judgments,  not  assumptions 
unless  they  are  used  to  establish  boundary  conditions  in  a  “what  if’  analysis.  Source 
appropriately. 

•  Target’s  Environment  and  Context:  As  we’ve  said  repeatedly,  the  meaning  of 
information  depends  on  the  context;  each  individual  provides  the  context.  Your 
target’s  decision  and  actions  take  place  within  their  environment,  and  their  thinking 
is  affected  by  the  individual  context  that  they  bring  to  a  situation.  As  you  convey 
your  assessment  about  the  target,  you  must  also  convey  selected  information  about 
the  target’s  environment  and  context  to  aid  your  audience  in  understanding  the 
target.  Consider  including  in  your  product  aspects  about  the  target’s  context  that 
influenced  your  beliefs  about  the  originator’s  (i.e.,  the  target’s)  meaning  of 
infonnation.  In  the  FAC  we’ve  often  discussed  that  the  context  is  dynamic.  Your 
audience  needs  to  understand  the  changing  context  associated  with  the  target’s 
current  situation.  Understanding  how  dynamic  the  situation  is  provides  the  audience 


197  Schum  p.  15. 

198  Schum  p.  5. 

199  Schum  pp.  18-19. 
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perspective  on  how  things  are  changing  and  how  fast.  As  Schum  says,  “...discovery 
and  inference  in  intelligence  analysis  and  elsewhere  take  place  in  a  non- stationary 
world.  The  world  changes  as  we  are  attempting  to  explain  what  is  going  on  in  it.”200 
The  same  is  true  for  your  audience’s  decision  making  and  planning,  i.e.,  the  world 
changes  as  they  are  preparing  to  make  decisions  and  take  actions.  Consequently, 
conveying  information  relevant  to  the  audience  about  the  changing  context  is 
essential. 

•  Uncertainties:  Ensure  you  address  uncertainties  that  are  important  from  the 
audience’s  perspective,  given  what  they  may  do  with  your  assessment.  Think  about 
the  audience’s  purpose  and  the  potential  decisions  or  actions  they  may  take. 
Consider  which  uncertainties  are  essential  to  convey  from  this  perspective.  These 
uncertainties  may  include  those  about  your  sources,  evidence,  beliefs  about  the 
current  status  of  the  target,  and  your  predictions.  Be  clear  about  the  type  of 
uncertainty  you  are  discussing  (e.g.,  differentiate  uncertainty  about  the  reliability  of 
a  source  from  uncertainty  due  to  missing  evidence  from  uncertainty  about  a 
judgment,  etc.) 

How  should  I  organize  the  narrative? 

There  is  no  prescribed  order  or  formula  for  organizing  narratives  that  is  effective  for  all  products. 
Present  your  narrative  in  a  logical  sequence.  That  logical  sequence  should  be  the  best 
explanatory  account  of  the  situation.  It  may  or  may  not  be  in  temporal  order.  Keep  in  mind 
the  elements  of  the  narrative  must  be  interwoven  as  appropriate  based  on  many  factors,  including 
your  audience’s  purpose,  your  purpose,  and  the  subject  matter.  Some  possible  approaches 
include: 

•  judgments  that  reflect  your  beliefs  followed  by  evidence  (the  most  supporting  first), 
followed  by  contextual  information; 

•  most  important  judgments  that  reflect  your  beliefs,  followed  by  judgments  of 
secondary  importance,  along  with  relevant  evidence-based  speculations  and 
interpretations; 

•  most  important  judgments  that  reflect  your  beliefs  with  evidence,  secondary 
judgments,  and  context  information. 

Why  is  the  narrative  assessment  important? 

The  elements  of  the  narrative  assessment,  framed  for  the  audience’s  perspective,  substantiate 
your  concise  assessment.  By  providing  your  knowledge  on  each  of  the  elements  of  the  narrative 
assessment,  you  enable  the  audience  to  understand  what  you  now  believe  about  the  target.  In 
addition,  the  supporting  evidence  and  chain  of  reasoning  that  you  provide  will  persuade  the 
audience  to  share  those  beliefs.  By  telling  a  clear,  effective  story  that  conveys  the  uncertainties 
associated  with  your  assessment,  the  audience  can  evaluate  the  potential  risks  of  decisions  and 
actions  taken  based  on  your  assessment.  The  narrative  also  provides  sufficient  knowledge  for  the 
audience  to  understand  the  people,  places,  and  things  that  are  relevant  to  their  interests,  decision 
making,  and  actions.  Consequently,  it  provides  the  foundation  of  knowledge  for  the  audience  to 


2UU  Schum  pp.  18-19. 
201  Schum  p.  29. 
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formulate  options  for  decision  making  and  action  that  are  needed  to  fulfill  the  audience’s  mission 
and  purpose. 

12.7  Other  Hints  for  Effective  Customer  Products 

The  audience  most  likely  does  not  need  to  know  everything  you  know.  If  you  carefully 
conveyed  your  concise  and  narrative  assessments,  guarantee  that  every  word  you  include  in  the 
customer  product  contributes  to  your  purpose  of  sharing  meaning  about  the  target  with  your 
audience.  This  results  in  an  organized  and  focused  communication  that  the  customer  can  use  to 
make  decisions  or  take  other  actions,  fully  understanding  relevant  uncertainties.  Precision  when 
writing  all  the  elements  of  the  narrative  is  essential.  It  will  also  help  you  not  turn  a  short  story 
into  a  long  story.202 

Don’t  over  argue.  As  S.  D.  Stark  counsels  attorneys  writing  legal  arguments,  “every  point  does 
not  have  to  be  the  essential  point,  and  every  contention  is  not  of  equal  value.”203  One  indicator 
that  you’ve  over-argued  in  your  narrative  may  be  that  you’ve  made  a  short  story  into  a  long 
story.  Not  all  points  you  made  in  your  analysis  representation  should  be  included  in  the 
customer  product.  Focus  on  the  compelling  points  given  your  audience’s  purpose  and  interests. 
Write  to  cover  potential  questions  that  you  anticipated  based  on  your  knowledge  of  the  customer, 
their  purpose,  and  beliefs. 

It  may  be  important  for  your  audience  to  know  some  information  about  the  standpoint  from 
which  you  approached  the  analysis.  For  example,  you  may  want  to  share  with  your  audience 
where  you  are  in  your  overall  understanding  of  the  target  since  your  analysis  is  time  and  context 
dependent  and  that  nature  of  your  arguments  may  change  during  various  stages  of  analysis  and  in 
light  of  new  information  received.  Over  time,  your  purpose  and  objectives  for  doing  the  analysis 
may  have  changed  as  well.  You  should  consider  if  such  a  change  in  purpose  is  relevant  for  the 
customer  to  know.  It  may  also  be  useful  for  your  audience  to  know  if  your  analysis  was  time 
constrained  in  such  a  way  that  impacted  your  analysis  relative  to  the  customer’s  interests.204 

According  to  Schum,  the  “. .  .literature  suggests  that  stories  are  wonderful  vehicles  for  “cheating” 
in  the  process  of  persuasion. . .”  Be  cognizant  that  you  may  be  inclined,  for  various  reasons,  to 

temper  or  adjust  your  assessment  if  you  think  that  it  will  not  be  well  received  by  the  audience. 
The  influences  to  do  so,  both  self-generated  and  generated  by  others  can  often  be  quite  subtle. 

For  instance,  you  may  be  uncomfortable  delivering  an  assessment  that  contradicts  a  high  level 
customer’s  beliefs  and/or  policies.  It  can  be  tempting  to  tell  a  short  story  that  is  a  good  story,  but 
not  one  well  supported  by  the  evidence.  This  is  why  Schum  refers  to  narratives  as  “necessary 
but  dangerous.”  Carefully  compare  your  draft  narrative  to  the  assessment  you  wrote  for  the 
analysis  representation  to  ensure  that  you  have  not  modified  the  essential  content  of  the 
assessment  based  on  what  you  now  understand  about  the  customer. 


202  Schum  p.  29. 

203  Stark,  S.D.  Writing  to  Win:  The  Legal  Writer:  The  Complete  Guide  to  Writing  Strategies  that  Will  make  Your 
Case—  and  Win  it,  Broadway  Books,  New  York,  NY,  2000,  p.  73. 

204  Schum  pp.  18-19. 

205  Anderson  p.  262. 

206  Schum  p.  29. 
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HINT  FOR  COLLABORATIVE  PRODUCTS 

Stark  has  a  rule  for  writing  persuasive  documents  for  legal  cases  that  is  particularly 
applicable  to  writing  collaborative  intelligence  analysis  products.  “Don’t  divide  the 
drafting  of  a  document  among  many  writers.”  [Stark,  Writing  to  Win,  page  14-15] 

Stark’s  discusses  that  many  writers  produce  a  draft  with  no  one  style.  He  says  that  if 
there  are  multiple  writers  they  must  have  worked  together  for  years  and  have 
compatible  styles. 

We  recommend  Stark’s  rule  be  applied  to  improve  your  narrative  in  an  additional  way. 
Multiple  writers  can  also  have  a  negative  impact  on  the  substantive  content  and 
persuasive  impact  of  your  message.  Your  message  must  be  coherent,  logically 
consistent,  and  presented  as  a  meaningful  totality  or  whole.  In  practice,  during  analysis 
there  are  various  activities  working  with  other  analysts  referred  to  as  collaboration. 
Similarly,  there  are  different  activities  referred  to  as  collaboration  that  take  place  during 
product  writing,  ranging  from  cutting  and  pasting  others’  inputs  to  fully  and  jointly 
writing  the  product.  From  a  cognitive  perspective,  substantive  coherency  and 
construction  of  an  effective,  persuasive  message  focused  on  your  audience’s  needs  is 
best  attained  if  you  have  one  or  few  writers  as  described  in  the  following.  You  want  to 
achieve  the  state  where  you  collaborate  so  closely  that  sometimes  you  lose  track  of  who 
has  written  which  part  of  the  text. 

“In  every  sense  of  the  word  the  book  is  coauthored.  We  taught  together,  class  by  class, 
and  have  written  together,  chapter  by  chapter.  It  has  been  a  long  process.  We  each 
wrote  half  the  chapters  in  our  first  draft,  then  swapped  them  for  redrafting  with  the  rule 
that  anything  could  be  changed.  Almost  everything  was.  We  had  a  further  rule  that  any 
changes  could  be  argued.  Some  were,  some  not.  We  carried  on  the  argument  by  means 
of  successive  drafts,  with  new  material  subject  to  the  same  rules.  There  were  so  many 
swaps  that  each  of  us  would  change  something  only  to  be  surprised  by  the  other’s  “OK 
with  me,  you  wrote  it.  ”  We  debated  sources  and  argued  out  interpretations.  Not  only 
can  we  no  longer  remember  who  first  wrote  what,  we  cannot  now  remember  who  first 
thought  what — or  even  who  first  found  what  when  we  jointly  researched  something.  We 
are  of  one  mind  and  (we  hope)  one  voice.  ”  [Thinking  in  Time,  Richard  Neustadt  and 

Ernest  May.] 

Achieving  this  state  is  not  always  possible  or  practical.  If  it  isn’t,  we  recommend  that 
one  analyst  draft  the  narrative  assessment,  followed  by  extensive  review  and  discussion 
with  collaborative  analysts. 

12.8  Critiquing  Your  Draft 

Now  that  you’ve  drafted  the  narrative  assessment,  you  should  go  back  and  review  your  draft 
concise  assessment.  Check  to  see  that  the  major  points  of  your  message  are  adequately 
supported  by  your  narrative.  If  they  aren’t,  make  the  necessary  adjustments  to  the  narrative 
assessment.  You  may  also  find  that  you  need  to  adjust  the  language  in  your  draft  concise 
assessment  so  that  it  is  better  focused,  based  on  the  audience’s  interests  and  the  elements 
contained  in  your  narrative. 
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When  you  think  you  are  satisfied  with  both  the  concise  assessment  and  the  narrative,  and  the 
relationship  between  the  two,  there’s  one  last  step.  Review  your  customer  product  one  last  time, 
using  the  questions  in  Appendix  1 .  If  necessary,  make  adjustments  to  the  product. 

12.9  Summary 

This  section  provides  infonnation  to  address  the  final  cognitive  challenge:  communicating  your 
assessment  to  a  customer  or  audience.  This  audience  can  be  internal  or  external  to  the 
intelligence  community,  an  individual  or  an  organization,  or  an  actual  or  prospective  customer. 

To  communicate  effectively  with  your  audience  means,  you  must  be  able  to  share  the  meaning  of 
your  assessment  with  your  audience.  This  requires  that  you  do  additional  analysis  of  your 
audience. 

To  develop  your  customer  product,  you  will  rely  extensively  on  the  writing  in  your  analysis 
representation,  which  represents  all  your  prior  work  in  the  FAC  on  the  intelligence  issue.  The 
customer  product  has  two  major  components:  the  concise  assessment  and  the  narrative 
assessment.  Both  must  be  framed  in  terms  of  your  audience’s  characteristics  and  needs. 
Recommendations  and  hints  are  provided  to  aid  you  in  writing  an  effective  narrative.  If  you  are 
entirely  successful  in  sharing  your  meaning  with  your  audience,  your  audience  will  share  the 
beliefs  that  you  formed  about  the  target  and  use  those  beliefs  to  accomplish  their  mission. 

When  you  wish  to  instruct,  be  brief;  that  men's  minds  take  in  quickly  what  you  say,  learn  its 
lesson,  and  retain  it  faithfully.  Every  word  that  is  unnecessary  only  pours  over  the  side  of  a 

brimming  mind. 

-Cicero 

12.10  Supplement  1:  Questions  for  Critiquing  Your  Narrative207 

•  Is  your  story  coherent?  Is  it  presented  as  a  meaningful  totality  or  whole? 

•  Is  the  story  fully  articulated? 

•  Is  the  story  internally  consistent? 

•  If  you  were  unable  to  arrive  a  single  overall  belief  for  your  assessment,  have  you 
communicated  that  to  your  audience?  Given  how  the  audience  may  use  the 
assessment  you  provide,  have  you  communicated  what  is  important  for  them  to 
know  about  your  multiple  beliefs  and  the  strength  of  those  beliefs? 

•  Have  you  included  all  of  the  significant  evidence  that  provides  the  foundation  for 
your  beliefs  and  resulting  assessment?  Have  you  included  any  evidence  of 
consequence  which  does  not  support  your  assessment?  Is  the  evidence  appropriately 
sourced? 

•  Is  there  any  evidence  that  seems  to  conflict  with  the  story?  Have  you  described  that 
evidence  and  the  implications  from  the  customer’s  perspective? 

•  Are  there  any  elements  in  the  story  not  supported  by  the  evidence?  To  what  extent 
are  they  based  on  speculation?  Is  your  speculation  evidence-based?  Have  you 
clearly  articulated  where  you’ve  gone  beyond  the  evidence? 


207  Questions  adapted  from  Schum,  D.A.  Narratives  in  Intelligence  Analysis:  Necessary’  but  often  Dangerous, 
George  Mason  University,  Fairfax,  VA,  2005,  pp.  8-25  and  Anderson,  T.,  Schum,  D.A.  &  Twining,  W.L.  Analysis 
of  Evidence,  2nd  edn,  Cambridge  University  Press,  Cambridge,  UK,  2005,  pp.  281-282. 
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Have  you  summarized  the  extent  to  which  the  evidence  supports  the  narrative? 

Does  the  story  sneak  in  expressly  or  by  innuendo  any  irrelevant  evidence? 

If  you  do  not  have  complete  evidence  relative  to  the  situation  of  interest,  have  you 
filled  in  gaps?  Have  you  clearly  articulated  where  that  is  the  case? 

Insofar  as  the  story  goes  beyond  the  evidence,  is  it  supported  by  plausible 
assumptions? 

Have  you  clearly  identified  those  assumptions?  (Keep  in  mind  the  FAC  definition 
of  assumptions) 

If  your  audience  already  has  their  own  story  about  the  issue  which  is  different  from 
yours,  have  you  thought  about  what  evidence  would  be  necessary  to  support  the 
audience’s  story?  Have  you  clearly  identified  whether  you  have  that  evidence  or 
not? 

Have  you  adequately  addressed  audience  questions  that  you  can  anticipate  given  the 
differences  in  the  audience’s  mindware  and  yours  relative  to  the  intelligence  issue? 

Based  on  your  audience  analysis,  does  your  story  address  questions  that  you 
anticipate  the  audience  may  ask  about  the  credibility,  relevance,  and  force  of  your 
evidence?  Does  your  story  address  and  properly  convey  how  your  assumptions  and 
the  judgments  of  others  were  used  in  the  assessment? 

Have  you  declared  important  contextual  and  time-related  elements  that  have 
implications  for  your  analysis?  For  example,  your  analysis  is  time  and  context 
dependent  and  the  nature  of  your  arguments  may  change  during  various  stages  of 
analysis  and  in  light  of  new  information  received. 

If  your  purpose  for  the  analysis  changed  during  the  course  of  your  analysis,  have 
you  considered  if  that  change  in  purpose  is  relevant  for  the  audience  to  know  about? 
If  it  is,  have  you  articulated  that  in  your  narrative? 

Have  you  appropriately  addressed  the  sources  of  evidence  queried?  Is  evidence 
sourced? 

Do  you  have  unanswered  questions  given  the  evidence  presently  available?  Have 
you  communicated  that  no  evidence  was  available  from  all  the  sources  you  queried 
to  answer  some  of  your  questions  about  the  target? 

If  the  time  to  do  your  analysis  was  time  constrained  and  the  effects  on  those  time 
constraints  impacted  your  analysis,  have  you  communicated  that  to  the  customer? 

Have  you  presented  your  narrative  in  a  logical  sequence?  That  logical  sequence 
should  be  the  best  explanatory  account  of  the  situation.  It  may  or  may  not  be  a 
chronology. 

Have  you  conveyed  your  story  so  the  dynamic  nature  of  the  situation  is  captured? 
You  and  the  audience  are  trying  to  make  sense  out  of  a  world  that  keeps  changing. 
Has  that  been  adequately  conveyed? 

Is  the  story  longer  than  it  needs  to  be?  Have  you  made  a  long  story  out  of  a  short 
story?  Is  this  because  you’ve  over  argued  your  story? 
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AAWE 

American  Association  of  Wing  Economists 

ACH 

Analysis  of  Competing  Hypotheses 

AFIAA 

Air  Force  Intelligence  Analysis  Agency 

AFRL 

Air  Force  Research  Laboratory 

AOR 

Area  of  Responsibility 

CBA 

Cost  Benefit  Analysis 

CIA 

Central  Intelligence  Agency 

CINC 

Commander  in  Chief 

COMINT 

Communication  Intelligence 

DIA 

Defense  Intelligence  Agency 

DNI 

Director  of  National  Intelligence  Agency 

ELINT 

Electronic  Intelligence 

FAC 

Framework  for  Analytic  Cognition 

FISINT 

Foreign  Instrumentation  Signal  Intelligence 

GEOINT 

Geospatial  Intelligence 

GIS 

Geographic  Infonnation  Systems 

HUMINT 

Human  Intelligence 

ICR 

Intelligence  Collection  Requirement 

IMINT 

Imagery  Intelligence 

IQ 

Intelligence  Quotient 

IS&R 

Information  Storage  and  Retrieval  System 

ISAR 

Interferometric  Synthetic  Aperture  Radar 

JMIC 

Joint  Military  Intelligence  College 

LIDAR 

Light  Detection  and  Ranging 

MASINT 

Measurement  and  Signature  Intelligence 

MSI 

Multispectral  Imagery 

MTI 

Moving  Target  Indicator 

NGO 

Non  Governmental  Organization 

OSINT 

Open  Source  Intelligence 

REM 

Rapid  Eye  Movement 

SAT 

Structured  Analytic  Techniques 

SECDEF 

Secretary  of  Defense 

SIGINT 

Signal  Intelligence 

SWOT 

Strengths,  Weaknesses,  Opportunities,  and 
Threats 

TS/SCI 

Top  Secret/Sensitive  Compartmented 
Information 

U.S. 

United  States 
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